1. 17

Original post is on gopher at gopher://sdf.org:70/0/users/rusty/Post06-against-complexity

  1.  

  2. 10

    This feels like it misses that a lot of this complexity of there because people like it, not just because of “fighting for attention”. Major things gopher lacks:

    • Being able to include images, as illustrations or to discuss them

    • Text that looks good on a wide range of devices. Hard word wraps assume a number of characters per line that don’t work for phones.

    • Communications that can’t be observed or modified by anyone who happens to be on the network path between you and the server.

    Only the last one is fundamentally complex, but these all provide real value.

    1. 1

      Images can be transferred by gopher but not displayed inline. I don’t think this is necessarily a bad thing: usually, when I see an image on the web, I have not wanted to see it, & to the extent that I do want to see such images, I would find it completely acceptable to have performed another operation to download and display it.

      Hard word wraps, certainly, are bad style and should not be used (on gopher or elsewhere). Every text display system is capable of character-wrapping on lines (if not word-wrapping). This is not a problem with gopher, but a problem with the people who write documents for gopher.

      Crypto is useful if (1) a MITM has a reason to modify content, or (2) a MITM has a reason to intercept content. In the absence of stuff like authentication & authorization (stuff bolted onto the side of HTTP by NetScape), gopher documents are mostly static & gopher users are practically (if not reliably) anonymous – in other words, no secret information should ever be transmitted over the protocol, nor should information ever be tailored to individual users in any way. (After all, gopher is so stateless that it closes the socket after writing a response.) Despite some folks trying really hard to bolt crypto onto gopher, I hope they do not meet with success, because the inability to target users or perform commerce on gopher is what keeps it usable as a hypertext system.

      The complexity of the web is not because ‘people like it’ but because features were deemed useful for potential profit centers & added without thinking the consequences through. Any system built organically by beginners will eventually end up looking like the web.

      Anyway, I don’t think the primary thrust of this essay is promoting gopher. Instead, gopher is an example of how to get 90% of the desirable functionality with less than 1% of the code (and less than 1% of the undesirable stuff).

      My own (very strong) take: HTML (all of it) is a bug; HTTP (most of it) too; CSS is a collection of bugs intended to work around HTML’s limitations without actually fixing them, and ends up merely adding complexity. There is no utility in preserving a knot of technical debt this way. The features added largely should either not exist or should have been provided by already-existing facilities (that could not be used because of problems introduced by poor design).

      1. 2

        I want to focus on crypto, because that’s both the most complex feature we’re talking about and the one with the strongest case for it. Consider the case where gopher or something else similarly simple succeeds to the point where it’s an important protocol that a lot of information people care about is transmitted across. Sure, responses are not individually tailored, which means the information shouldn’t be secret, but there are at least two ways you still need crypto:

        • Without it other people can tell what you’re reading. You should be able to read things without your ISP/coffee shop/government knowing.

        • Without it other people can modify the content. This includes the range from public WiFi inserting ads into responses (simple text ads would still be commercially viable to inject if gopher/etc took off) to governments modifying news coverage or editorials to support their position.

        1. 1

          That’s true, but consider possible threat models. Why do people care what you’re reading? Why do people want to change the content?

          In the absence of commerce & personalization, vast third party commercial enterprises have no reason to do drag-net surveillance (they don’t care what you’re reading because they don’t know who you are) or to change data (they don’t know who you are or what you’ve been reading, so they have no reason to believe they can improve revenue by changing ads, nor does inserting ads make as much economic sense).

          In other words, your threat model is substantially less vague: you only need to care about secrecy if you personally know of specific organizations with a specific interest in specifically you, & you only need to care about anti-tamper if you specifically know a MITM has reason to change your stream (to feed you modified information for political reasons or something).

          If either of those things are true, it makes sense to use crypto for gopher. The appropriate way is probably to use ssh tunneling or tor, both of which already work & require no changes to the protocol.

          1. 3

            You’re forgetting authoritarian governments who have a vested interest in controlling the information their citizens consume.

            1. 1

              An authoritarian government is a known specific organization acting as a MITM with reason to change your stream. I intentionally specified the threat model to include that. Even so, authoritarian governments don’t want to intercept or modify all content – only certain types that are generally predictable.

            2. 2

              That’s true, but consider possible threat models. Why do people care what you’re reading? Why do people want to change the content?

              To inject advertising and make an extra few cents. There are already instances of ISPs doing this.

              1. 1

                People barely pay for targeted advertising (like, a couple cents for a few hundred click-throughs – not even impressions). What makes you think anybody will consider it viable to pay for advertising to be injected on a platform that supports neither tracking nor click-through metrics.

                1. 3

                  People barely pay for targeted advertising (like, a couple cents for a few hundred click-throughs – not even impressions).

                  This sounds way off. Quickly searching I see AdStage claiming “In Q1 2018, advertisers spent, on average, $2.80 per thousand impressions (CPM), and $0.75 per click (CPC). The average click-through rate (CTR) on the GDN was 0.35%.” Those are in-line with what I’d expect to see, and would be $75 for a hundred click-throughs, not $0.02. And this is for a mixture of personalized and unpersonalized ads, since it’s an average across the GDN.

                  There’s also content-based targeting: inserting ads for credit cards into someone’s post on how to choose a credit card would be very profitable.

                  What makes you think anybody will consider it viable to pay for advertising to be injected on a platform that supports neither tracking nor click-through metrics.

                  Advertisers bought ads before the internet, and still buy ads in untracked media: billboards, tv spots, radio, etc. Personalized ads are worth more, sure, but not so much that only personalized ads are worth running.

                  You also can still track clicks through URL decoration. This is like when you see an ad on the subway and it gives a url like www.example.com/subway.

                  1. 1

                    This sounds way off.

                    Numbers are from my experience running ads on blogs & personal websites since 2006. I have made a total of nine dollars and change from those ads, eight dollars of which come from a 2-month period in 2006 during which things were computed differently. I’ve had thousands of impressions and hundreds of click-throughs. Maybe my blogs were exceptionally hard to target or attracted only low-paying advertisers, or maybe AdStage’s metrics are biased by a handful of folks spending thousands on SEO or click farms.

                    Advertisers bought ads before the internet, and still buy ads in untracked media

                    You also can still track clicks through URL decoration.

                    Fair enough. I don’t think it’s a major risk, even in the case that gopher really takes off, on the grounds that the main lesson from adtech (and the falling price of personalized ads) is that even in the best-case scenario (micro-targeted ads with 100% tracking across sites), ads don’t work well enough to spend market rates on. That said, it’s not like people don’t regularly make terrible business decisions.

                    Tracking through URL decorations is definitely a thing, but I only really see it used in an influencer context (where somebody has a strong parasocial relationship with, say, a podcast host & so is disposed to include the tracking info when retyping a URL). With third party ad injection, impressions are unintentional on the part of the user and click-throughs are accidental. It’s very different from someone you trust recommending things to you. Techniques to turn accidental click-throughs into impulse purchases are less effective when you can’t make purchases through the same protocol, as well.

                    1. 1

                      Numbers are from my experience running ads on blogs & personal websites since 2006. I have made a total of nine dollars and change from those ads, eight dollars of which come from a 2-month period in 2006 during which things were computed differently. I’ve had thousands of impressions and hundreds of click-throughs.

                      Hundreds of click-throughs with thousands of impressions seems much higher than I’d expect; normally you see 0.1% to 1% as many clicks as impressions?

                      I’ve also had ads on my personal sites, going back to late 2010, though I had them turned off for a while 2015-2018. Checking AdSense, I see a CPM of $1.79, and a CPC of $0.54. These are also after AdSense’s 32% cut, which means advertisers are paying more like $0.80 CPCs.

                      (Disclosure: I work for Google, though I’m speaking only for myself)

                  2. 1

                    Because it would be so cheap to inject content into text as to make it worth it regardless of targeting or tracking. Look at spam email. Or how Google displayed ads in GMail based on text in the messages. That’s a level of targeting that might give it value.

                    You’d have to be part of the connection chain like an ISP or WiFi provider, unlike email where anyone can send mail to anyone. But “free” WiFi could have the ability to do this and someone might think it worth doing. A lot of stores (or places otherwise selling something) provide WiFi and would have an incentive to advertise in any plain text protocol if the tools were available to them.

                    1. 1

                      If you’re a large ISP & you have enough of a monopoly that you have no risk of losing customers by alienating them with injected ads, you have the potential to inject ads & perhaps the motive. (And, Comcast famously did this with targeted ads, replacing already-existing ads with their own.) But, that’s a lot more effort for less return than simply increasing your prices and lowering your quality of service.

                      If you’re using gopher and your ISP has started doing this, I would recommend using an encrypted tunnel. I do not think gopher ought to have out-of-the-box encryption. Gopher is nice because even a newbie programmer can implement a gopher server or a gopher client in a couple lines of code. Using a tunnel maintains that simplicity, while building encryption into the protocol means only crypto experts can write gopher implementations anymore.

                      1. 2

                        If you’re using gopher and your ISP has started doing this, I would recommend using an encrypted tunnel.

                        I agree. I think something like Tor is probably the best solution as it solves multiple problems such as protecting your IP, protecting your DNS queries, as well at protecting the data in transit. And as long as it’s not a hidden service only, it can be opted out of to maintain backwards compatibility or remain light weight if someone doesn’t want/need the security.

        2. 4

          When users fully understand HOW their programs work, they retain their control over the computing experience.

          Here’s the thing: most people don’t care. I’ve wanted to believe that “control over the experience” matters for a long time, and part of my malaise with tech lately has been coming to grips with the fact that for likely the majority of people this is simply a non-issue. This is why environments that offer that kind of control don’t interoperate so well with ones that don’t.

          [Gopher’s] simplicity kept it alive: it requires little knowledge to set-up & it requires little in terms of resources. Gopher has been left unscathed […]

          Not too be too harsh, but this is because no one uses it. Here’s a Gedanken-Experiment: suppose it was Gopher that got popular and not the Web. Do you think Gopher would be as simple today?

          The Web isn’t complicated because developers were negligent, it’s complicated because it attracted a lot of attention and people wanted it to do more. And fast. I agree that it’s over-complicated and it drives me nuts. But I think it’s very wrong to believe that all the stuff we do now that we weren’t doing in the same way 20-30 years ago (and that’s a lot) is boiled down to something as elegant as a pipeline command in the shell. It’s good to fight complexity; it’s folly to think everything is simple.

          1. 1

            Here’s the thing: most people don’t care. I’ve wanted to believe that “control over the experience” matters for a long time, and part of my malaise with tech lately has been coming to grips with the fact that for likely the majority of people this is simply a non-issue. This is why environments that offer that kind of control don’t interoperate so well with ones that don’t.

            People care that their stuff works the way they want it to. They want reasonable defaults, and they want bugs fixed. I don’t think professional software developers can provide either of these things at scale, because the variety of applications is too wide for a professional class to cater to (and thus, more often than not, reasonable defaults aren’t reasonable for all users & correct behavior for one user is a bug for another).

            There are a few alternatives to making a small professional class of devs write a lot of special-casing code for handling a huge variety of actual user cases:

            1. handle a couple of user cases that are more or less representative, and have the rest of the users suck it up or invent their own workarounds based on a vague & warped understanding of how the program is supposed to work. (This is what we do now.)
            2. don’t handle user cases at all, and instead focus on the hard stuff while making the easy stuff easy enough for users to roll their own solutions. (This is what web development was like before it became professionalized, and more or less corresponds to what physicists and linguists end up doing with shell scripts.)

            I think option #2 is basically preferable for everybody (even though it means that the demand for code monkeys basically disappears), but it requires us to make the tools simpler and more available. Already, in many cases, the easiest way for a non-technical end user to get the behavior they want is to learn to code, but we could make things easier for them by designing systems so that making the kinds of shallow changes end users tend to want can be performed with only a shallow understanding of both the language & the rest of the code (i.e., to make most changes desired by end users tantamount to slotting together existing calls).

            suppose it was Gopher that got popular and not the Web. Do you think Gopher would be as simple today?

            I think you’re right that Gopher would have undergone the same change as the Web did, but I think you’re wrong that developers are not basically complicit in this, or that it’s inevitable. We can learn from history, and refuse to implement only-shallowly-desirable features while making sure whatever new features we implement are consistent & scale.

            The sin we committed with the web is naivete: we assumed that, no matter how unstructured and ad-hoc we were, complexity would never add up. We now know that we were wrong about that. We can’t save the web, because the technical debt has been ossified in standards, but the next time we build a hypertext system we can endevour to manage complexity properly.

          2. 2

            Not sure what a misunderstanding of Marie Kondo has to do anything, but sure, go off on one about charlatans… This just feels like reactionary bullshit (rather than anything revolutionary) with a total failure to understand what has changed about the world outside the computer and what people’s relationships with computers is.

            1. 1

              What, in your opinion, has changed about people’s relationship to computers since 1990 that justifies the complexity of, say, web browsers?

              1. 4

                The fact that on the web, you can buy almost any product you want, transfer your money around, plan and book your next vacation, (in some countries) you can register a car, your children and even yourself, or in other words, the “world wide web” has become far more integral and it’s requirements have grown from becoming a markup protocol to one of the main communication media of our global society.

                I’m not sure if you would say it “justifies” it, but I do think that it explains it.

                1. 1

                  Absolutely. We’re using the web for a wide variety of things that it’s not meant to do, and this required sticking a bunch of extra crap on top of it, and the people who stuck the extra crap on top weren’t any more careful about managing technical debt than the people who designed the thing in the first place.

                  There’s a need for secure computerized mechanisms for commerce, exchange, and identification management. Using web technologies to produce stopgap solutions has lowered the perceived urgency of supplying reliable solutions in that domain. It’s put us back twenty-five years, because (at great cost) we’ve made a lot of stuff that’s just barely good enough that it’s hard to justify throwing it all out, even when the stuff we’ve built inevitably breaks down because it’s formed around a set of ugly hacks that can’t be abstracted away.

                  1. 3

                    But that’s the thing – the web you talk of is little more than ruins that people like me or you visit and try to imitate. It has become irrelevant that the web “wasn’t meant” for most of the things it is being used today, it has been revamped and pushed towards a direction where it retroactively has been made to seem as though it had been made for it, it’s history rewritten.

                    It’s not clean, but with the prospect of a browser engine monopoly by groups like Google, and their interests, this might get improved on for better or worse.

                    The point still stands that I have made elsewhere that the web has been the most fertile space for the real needs of a computer users, both “consumers” as well as marketing and advertisement firms – one can’t just forget the last two, and their crucial role in building up the modern web into something more or less workable.

                    If we had wanted the web to be “pretty” and “clean” I think it would have had to have grown a lot more organically, slower and controlled. Instead of waiting for AOL, yahoo and gmail to offer free email, this should have been locally organised, instead of having investors thrown into a space where the quickest win, technologies and protocols should have had time to ripen, instead of having millions of people buying magical computer boxes that get better with every update, they should have had to learn what it is they are using – having have said that, I don’t believe this would have been in any way practical. Beauty and elegance don’t seem to scale to this level.

                    1. 1

                      The lack of cleanliness isn’t merely a headache for pedants & an annoyance for practitioners: it makes most tasks substantially more difficult (like 100x or 1000x developer effort), especially maintenance-related tasks, and because the foundations we’re building upon are shoddily designed with unnecessary abstraction leakages & inappropriate fixations, certain things we need to do simply can’t be done reliably. We’re plugging the holes in the boat with newspaper, and that’s the best we can do.

                      Beauty and elegance don’t seem to scale to this level.

                      Scaling is hard, and I don’t expect a large system to have zero ugly spots. But, complication tends to compound upon itself. For the first 15-20 years of the web, extremely little effort was put into trying to make new features or idioms play nice with the kinds of things that would come along later – in other words, everything was treated like a hack. Tiny differences in design make for huge differences in complexity after a couple iterations, and everything about the web was made with the focus on immediate gains, so as a result, subsequent gains come slower and slower. Had the foundational technologies been made with care, even with the kind of wild live-fast-die-young-commit-before-testing dev habits that the web generation is prone to, the collapse we are currently seeing could have been postponed by 20 or 30 years (or even avoided entirely, because without the canonization of postel’s law and “rough consensus and running code” we could have avoided ossifying ugly hacks as standards).

            2. 2

              There’s so much wrong here it hurts. Let’s start with the idea that command lines are somehow more honest and have “no hidden state” - it’s utter bollocks. Command lines as they currently exist are entirely about hidden state and options and secret incantations that must be pre-memorised by the user before they actually try to do anything.

              The problem with software isn’t that “so-called programmers don’t know about pointers and cache lines” - the problem with software is that we haven’t updated the interfaces between its various parts since the 1960s, and that the world of software authorship (plain text files full of jargon) is completely detached from the world of software execution.

              1. 5

                The VMS command line was very discoverable and consistent (then again, all the programs were written by a single company). For modern examples of command lines, the CLI on professional routers are incredible. At any point you can hit TAB or ? and get a list of possible completions from that point in the command line, and you only need to type as much to make the selection unambiguous (for example, on Cisco, “sh int” and “show interface” do the same thing).

                And despite source code being in text, I can still work with code written in the 60s and 70s. Try loading a Microsoft Word document from 1986 …

                1. 1

                  It’s not that I think all command lines have to be terrible, or even that all the ones we have are. But there’s little to no work taking place to improve them. As for word, I think the problem is poor documentation and proprietary formats, rather than text vs binary. And yes, old word files are terrible, given they’re basically core dumps :)

                  1. 1

                    At any point you can hit TAB or ? and get a list of possible completions from that point in the command line

                    zsh also supports this, which is fairly impressive considering it’s doing so for an arbitrary set of binaries.

                  2. 3

                    Command lines as they currently exist are entirely about hidden state and options and secret incantations that must be pre-memorised by the user before they actually try to do anything.

                    I hear this take a lot, and I can’t understand it at all. As awkward as the command line can sometimes be, it’s a learnable system: it exposes all its facilities and parameters through a roughly-consistent interface (i.e., no ‘hidden state’ because everything about it is viewable and changable by the user), has built-in documentation, and users can learn incrementally rather than memorizing anything because there’s no boilerplate to speak of. There’s only slightly more front-loaded learning than a GUI, in exchange for a lot more control: any literate person can become proficient in using a command line to perform relatively complex operations, without ‘learning to code’. It’s the best user interface 1969 has to offer, and the biggest problem with it is that after fifty years it has no widely-available competitors.

                    It’s possible to make an interface that improves substantially on a unix CLI, keeping the level of automatability & control while making inter-process communication more general, syntax more consistent & easier to learn, and integrating support for rich media. PARC did it in the 70s. Smalltalk-style systems didn’t take off, & the reason seems to be their limited support for binaries (which have some performance benefits mostly obviated by JIT, and have opacity, which we obviously don’t care much about anymore since we’ve moved much of our ecosystem to web apps).

                    It’s not really surprising that somebody who doesn’t feel up to implementing a Smalltalk-style system would bring up command line interfaces as an example of handing power & access to the end user. The difficulty of learning to use a command line is vastly overstated, & almost all of it is the result of attempts at maintaining backward compatibility with older, organically-developed, competing systems, rather than essential parts of the interface paradigm.

                    1. 3

                      secret incantations that must be pre-memorised by the user before they actually try to do anything

                      Your problem with the command line is it requires learning?

                      1. 3

                        I think that there is a problem with discoverability in CLIs. As I’ve said many times, the text-primary interface (ie. everything is a char *) necessarily makes it harder to maintain a uniform interface, and also limits the user in reducing/constructing everything to/from text streams.

                        Add to that the historical baggage of shells, all the environmental variables that can be set and mess up the internal working of some library or dependency, the “don’t say anything if nothing goes wrong” mindset of the unix philosophy (which makes sense for typewriters, but isn’t required for modern graphical displays), the hacks upon hacks that let us even use the same tools that were thought of in the 70’s somewhat usefully, etc.

                        The CLI has problems that people often mythologise instead of facing.

                        1. 1

                          The CLI can be made more discoverable, but it has better discoverability than basically all GUI environments (especially if you’ve got a modern shell, where tab completion works nicely). UNIX shells tend to be bogged down under the weight of backward compatibility and technical debt (which is why zsh’s default configuration is borderline unusable), but that’s not the fault of the command line model.

                          More problematic is the way that the shell doesn’t go all the way down – the interactive shell layer just manipulates individual tools, usually written in C, and even very good shell scripters have to put in a lot of effort if they want to also learn C. Which means that, at a certain point (actually pretty early), users run into a wall where they need to do a bunch of extra learning if they want to change the behavior of a certain class of tools, & not all of that information is available from the shell’s help system.

                    2. 1

                      Just a nitpick, I’m not sure if “Against complexity in computing” is the best title, since the first thing I thought of was complexity theory.

                      1. 1

                        ‘Complication’ would probably be a better word, yeah.