1.  

    Neat! Asking too much to add some affordances for hitting a running service via /debug/pprof/xxx? I ask because this is overwhelmingly how I use the tool.

    1.  

      I don’t see why not. This was just 10 minutes worth of work I figured I’d share with everyone, especially to lower the bar to get into pprof-ing.

      I also made this public to get feedback like this. I’ll throw up an issue on the repo, unless you would like to?

    1. 9

      The nice thing about Go is that when it is verbose, like for the list deletion example, it highlights that the computer is doing more work. If you are constantly deleting items in the middle of an array (or if you’re doing this at all), it might not be the best choice of data structure.

      1. 30

        This sounds like a post hoc rationalization to me. You can always hide arbitrary computations behind a function call.

        I believe this is explained by the lack of generics, and I predict that, if/when generics are implemented, go stdlib will gain a bunch of slice-manipulating functions a-la reverse.

        1. 2

          Which is probably one of the stronger arguments for genetics.

          1. 2

            This sounds like a post hoc rationalization to me.

            This was always pretty reliably cited as the motivation for a lot of design decisions, back when Go was just released.

            You can always hide arbitrary computations behind a function call.

            Yes, but one nice thing about Go is that functions are essentially the only way to hide arbitrary computation. (And, relatedly, the only mechanism that Go has to build abstraction.) When you’re reading code, you know that a + b isn’t hiding some O(n²) bugbear. That’s valuable.

            1. 3

              Yup, I agree with the general notion that reducing expressive power is good, because it makes the overall ecosystem simpler.

              But the specific example with list manipulation is “a wrong proof of the right theorem”, and this is what i object to.

              Incidentally, a + b example feels like a wrong proof to me as well, but for a different reason. In Go, + can also mean string concatenation, so it can be costly, and doing + in a linear loop can be accidentally quadratic. (I don’t write Go, might be wrong about this one).

              1. 1

                In Go, + can also mean string concatenation, so it can be costly

                How do you mean? String concatenation is O(1)…

                doing + in a linear loop can be accidentally quadratic. (I don’t write Go, might be wrong about this one).

                How’s that?

                1. 2

                  String concatenation is O(1)…

                  Hm, I don’t think that’s the case, seem to be O(N) here:

                  
                  14:48:36|~/tmp
                  λ bat -p main.go 
                  package main
                  
                  import (
                      "fmt"
                      "time"
                  )
                  
                  func main() {
                      for p := 0; p < 10; p++ {
                          l := 1000 * (1 << p)
                          start := time.Now()
                          s := ""
                          for i := 0; i < l; i++ {
                              s += "a"
                          }
                          elapsed := time.Since(start)
                          fmt.Println(len(s), elapsed)
                      }
                  }
                  
                  14:48:40|~/tmp
                  λ go build main.go && ./main 
                  1000 199.1µs
                  2000 505.49µs
                  4000 1.77099ms
                  8000 3.914871ms
                  16000 14.675162ms
                  32000 49.782358ms
                  64000 182.127808ms
                  128000 661.137303ms
                  256000 2.707553408s
                  512000 11.147772027s
                  
                  1. 3

                    You’re right! O(N). Mea culpa.

          2. 40

            it highlights that the computer is doing more work

            It seems strange to me to trust in the ritual of doing strange and obnoxius, bug-prone, programming to reflect the toil the computer will be burdened with. Doubly strange when append() is generic magical go function built-in which can’t be implemented in Go so it’s hard to even say, without disassembly and studying the compiler, what the code would even end up doing at runtime; I guess we can make very good educated guesses.

            1. 19

              IMHO, no performance concerns are valid* until a profiler is run. If it wasn’t run, the program was fast enough to begin with, and if it was, you’ll find out where you actually have to optimise.

              • Exception: if there are two equally readable/maintanable ways to write code, and one is faster than the other, prefer the faster version. Otherwise, write readable code and optimise when/if needed.
              1. 4

                I feel this statement is a bit too generic, even with the exception. Especially when you’re talking about generic stdlib(-ish) functions that may be used in a wide variety of use cases, I think it makes sense to preëmptively think about performance.

                This doesn’t mean you should always go for the fastest possible performance, but I think it’s reasonable to assume that sooner or later (and probably sooner) someone is going to hit some performance issues if you just write easy/slow implementations that may be “fast enough” for some cases, but not for others.

              2. 18

                But I want the computer to be doing the work, not me as human source code reader or, worse, source code writer.

                1. 1

                  Sure, but a language that hides what is computationally intensive or not, is worse.

                  1. 24

                    Source code verbosity is poorly correlated with runtime cost, e.g. bubble sort code is shorter and simpler than other sorting algorithms.

                    Even in the case of removing items from arrays, if you have many items to remove, you can write a tiny loop with quadratic performance, or write longer, more complex code that does extra bookkeeping to filter items in one pass.

                    1. 6

                      Then don’t build your language to hide what’s computationally expensive. Fortunately, most languages don’t do any sort of hiding; folding something like list deletion into a single function call is not hiding it - so this statement isn’t relevant to the discussion about Go (even if true in a vacuum). Any function or language construct can contain any arbitrary amount of computational work - the only way to know what’s actually going on is to read the docs, look at the compiler, and/or inspect the compiled code. And, like @kornel stated, “Source code verbosity is poorly correlated with runtime cost”.

                    2. 1

                      As a user, I don’t want the computer to be doing the work. It reduces battery life.

                      1. 10

                        If you’re referring to compilation or code-generation - that work is trivial. If you’re referring to unnecessary computation hidden in function calls - writing the code by hand is an incredibly bad solution for this. The correct solutions include writing efficient code, writing good docs on code to include performance characteristics, and reading the function source code and/or documentation to check on the performance characteristics.

                        As @matklad said, “You can always hide arbitrary computations behind a function call.”e

                  1. 2

                    Unit testing and continuous integration are now mainstream. In due time, I think property-based testing and fuzz testing will be mainstream.

                    1. 4

                      I don’t think that’s likely, because fuzz testing has a much narrower scope of applicability, and widening it takes nontrivial effort. Property and fuzz testing are powerful, but niche, tools.

                      1. 2

                        I think that technically it’s very doable: for example, GitHub Actions could run your fuzz tests for a certain number of minutes for free (or more if you pay), as in “go test -fuzz -fuzztime 10min ./...”. However, whether that’s actually a good idea is another question. Rob Pike asks some good questions on that issue that get to the heart of this: “the cost/benefit ratio [of fuzz testing] remains unclear” and “I will say that some of the bugs it finds are not worth fixing, and certainly not the cost of finding, although it’s very hard to make that case.”

                        As much as I like fuzz testing and think it’s good that Go’s making it easier, all testing does have a cost, and fixing the (sometimes very obscure) bugs it finds takes time. At my previous company I had a hard time convincing folks that a real Bobby Tables SQL injection bug on our production website needed to be prioritized.

                        1. 1

                          Unit testing also took nontrivial effort and required infrastructure for mocking and fixture. There will be infrastructure for property-based testing and fuzz testing too.

                          1. 2

                            Yeah, but I think that for most line of business applications, domain-oriented fuzzing represents an ~order of magnitude more work, for an ~order of magnitude less benefit, than unit testing.

                      1. 4

                        Fun problem!

                        Tracking both edge/delta and level/state triggers for counts data seems a bit needless, maybe? When I have systems like this I try to reduce to just the level triggers. Each thing that manages connections could emit timestamped and labeled integers representing current counts to some central place, or into some hierarchy that feeds to a central place. Total counts by label are then the sum of the most recent integers, back to some deadline, matching that label. Failures can just be dropped.

                        1. 2

                          I’m not very familiar with the edge/level terminology. I found this blog post enlightening, and thought others might too: http://gengnosis.blogspot.com/2007/01/level-triggered-and-edge-triggered.html.

                          This is a good point you make. In fact, we also count connections in the way you describe for a different use case: to calculate usage for billing/limiting purposes.

                          I wasn’t the one who designed the system described in the blog post, but I assume it grew organically from a simpler system (like the one I described at the start of the post). I think this organic evolution probably explains the deltas+cleanup approach. It may not be the simplest/elegant design, but the current implementation has worked well for us in practice.

                          I’m trying to think of a downside of the approach you suggest. The only things that comes time mind is there will be some lag between the connection counts on the nodes and the aggregated sum. There might also be some wasted events if the counts don’t change frequently. I think in practice these would not be significant issues for the requirements of our system.

                        1. 25

                          Go gives you the tools for concurrency, but doesn’t make them terribly easy - except to misuse.

                          Compare Erlang, which does make concurrency an essential part of its worldview, and makes building those systems less hassle-prone than Go.

                          1. 9

                            They use different models for concurrency, with different tradeoffs and different expressive power, but I don’t think Erlang or Go is generally better or worse than the other with regards to concurrency.

                            1. 5

                              Could you give us an example? I’m not familiar with Erlang

                              1. 5

                                Erlang concurrency is in two models: a low-level process (green thread, no shared state) that can send and receive messages using imperative code, and one of several high-level behaviors that encapsulate the error-prone sending and receiving semantics so the developer only has to implement a handful of callbacks that handle incoming messages that may or may not change the server’s state and may or may not expect a reply.

                                In a chat server, you’d model a connection as a gen_server or gen_statem that receives outgoing messages from TCP and incoming messages as a cast from a channel, and a channel as a gen_server that relays incoming messages to channel members.

                                http://erlang.org/doc/design_principles/gen_server_concepts.html has some specific information about behaviors.

                                https://learnyousomeerlang.com/the-hitchhikers-guide-to-concurrency is the first of several chapters in Fred Hébert’s excellent book about the low-level concurrency, and https://learnyousomeerlang.com/what-is-otp is the first of several about the high-level behavior-based model that I’m most familiar with.

                            1. 39

                              As (former) application author I find it very hard to sympathize with distro packagers if their opinions and the mentioned patches they make out of them continue to be responsible for a good chunk of bugreports that cannot be reproduced outside of their distro. Why should I cater to the whims of multiple Linux distros, what do I get out of putting more work into the product I already provide for free? Imagine Apple app store, on top of placing random restrictions on application submissions, added random patches to your application and is not sufficiently careful about which of them break the end user experience. That is what upstream maintainers have to deal with, and they don’t even get paid for it.

                              See also Linus on static linking and distro packaging.

                              Keep in mind that 1) this is literally only a problem on Linux distros or other third-parties repackaging applications and imposing their opinions on everybody 2) the actual context of this blogpost is that the author is mad at Python packages using Rust dylibs, it seems his sense of entitlement has not significantly improved since then.

                              1. 15

                                Ideally you don’t need to do anything except not make distro maintainer’s lifes harder.

                                If you absolutely want to provide your own binaries directly to endusers, as of 2020 there are things like Docker images and AppImage now so you can bundle what you need at this level.

                                So while we don’t have Linus dive tool in Void Linux yet, it looks easy to package and once that is done, the maintainers will take care to provide the newest version to the users, taking off work from you.

                                We also generally only try to patch build issues and security fixes that did not make it into a release yet. So often, users of our binary packages get fixed versions quicker than upstream.

                                1. 6

                                  Ideally you don’t need to do anything except not make distro maintainer’s lifes harder.

                                  I think the point of cognitive dissonance here is that what distro maintainers want often makes application developer’s lives harder. Dynamic linking doesn’t work well for many application developers, because libraries break even when they don’t change “major” versions: that’s just a fact of life. No software development process is perfect, and the application developer can’t reasonably test against every different patch that every different distribution applies to every different library. Being able to just drop a binary onto a machine and be confident it’ll work the same on that machine as it does on your own is a selling point of languages like Go and Rust.

                                  And if you want to change the libraries used for these languages it’s not exactly hard. Just change the go.mod or Cargo.toml to point to the library you want it to use, rather than the library it’s currently using, and rebuild.

                                  If you absolutely want to provide your own binaries directly to endusers, as of 2020 there are things like Docker images and AppImage now so you can bundle what you need at this level.

                                  Docker and co are worse for security than static linking. Packaging as a Docker container incurs all of the downsides of static linking, and also all of the downsides of any outdated packages in the image. Static linking only distributes the libraries you need: containers distribute effectively an entire OS minus the kernel (and also the libraries you need).

                                  Docker as a solution only makes sense if application developers want both dynamic linking and static linking; dynamic if you install it on the host, and effectively-static if you run it as a container. But the core issue is that many application developers do not want dynamic linking! If you do not want dynamic linking, static linking is better than using containers.

                                  1. 3

                                    I think the article confuses two separable things:

                                    • Bundling in the shipped product.
                                    • Provenance of inputs.

                                    The former is a problem in terms of computational resource, but not much else. If a program statically links its dependencies (or uses C++ header-only libraries, or whatever), then you need to redo at least some of the build every time there’s an update (and generally you redo the whole build because incremental builds after dependency updates are flaky). The FreeBSD project can rebuild the entire package collection (30,000+ packages) in under two days on a single machine, so in the era of cloud computing that’s a complete non-issue unless you’re running Gentoo on an old machine.

                                    The second is a much bigger problem. If there’s a vulnerability in libFoo, a distro bumps the version of libFoo. Anything that has libFoo as a build-time dependency is rebuilt. Security update fixed, we just burned some cycles doing the rebuild (though, in the case of static linking, possibly a lot fewer than we’d burn by doing dynamic linking on every machine that ran the program). If a program has vendored its dependency on libFoo, there’s no metadata conveniently available for the distribution that tells anyone that it needs to be rebuilt against a newer libFoo. It’s up to the program author to issue a security advisory, bump the library version, and so on. The distro will keep shipping the same library for ages without any knowledge.

                                    Things like Docker make this worse because they make it trivial to write custom things in the build that grab source from random places and don’t record the provenance in an auditable structure. If I have an OCI image, I have absolutely no idea what versions of any libraries I’m running. They may be patched by the person who built the container to avoid a bug that caused problems for a specific program and that patch may have introduced another vulnerability. They may be an old version from some repo. They may be the latest trunk version when the container was released.

                                  2. 5

                                    Securitywise Docker images are about as bad as static linking for the end user.

                                    1. 3

                                      Of course, but it’s easier on the entire supply chain in the 99.9% of cases there is no security problem.

                                      1. 8

                                        99.9%? do you mean 40%?

                                        https://www.techrepublic.com/article/docker-containers-are-filled-with-vulnerabilities-heres-how-the-top-1000-fared/

                                        “Over 60 percent of the top Docker files held a vulnerability that had a Kenna Risk Score above 330; and over 20 percent of the files contained at least one vulnerability that would be considered high risk under Kenna’s scoring model,” […] the average (mean) number of CVEs per container is 176, with the median at 37.

                                      2. 3

                                        Yes, and static linking has a known solution for security updates: the distro rebuilds from updated source.

                                        1. 3

                                          Yes, but this needs to be done so often and so broadly, that at least Debian just seems to do regular rebuilds of nearly everything every few weeks or so in unstable and declares that software written in Go has no proper security support in at least Debian 10 Buster and security updates will only be provided via the minor stable updates approximately every two months or so. Still a PITA and hence q.e.d.

                                      3. 5

                                        If you absolutely want to provide your own binaries directly to endusers

                                        You say this like it’s a method of last resort, but this is overwhelmingly how software authors prefer to package and distribute their applications. There’s lots of good reasons for that, and it’s not going to change.

                                        1. 1

                                          Ideally you don’t need to do anything except not make distro maintainer’s lifes harder.

                                          I don’t even need to do that. Again, I am providing free work here.

                                          If you absolutely want to provide your own binaries directly to endusers, as of 2020 there are things like Docker images and AppImage now so you can bundle what you need at this level.

                                          I am fairly sure if people started to do that at scale, distro maintainers would complain all the same as they do about static linking.

                                          So while we don’t have Linus dive tool in Void Linux yet, it looks easy to package and once that is done, the maintainers will take care to provide the newest version to the users, taking off work from you.

                                          You’re wholly missing the point with this sentence. The fact that we’re in a position where we need to build applications per-distro is unsustainable. There is very little work in building a static binary on any other platform.

                                          We also generally only try to patch build issues and security fixes that did not make it into a release yet. So often, users of our binary packages get fixed versions quicker than upstream.

                                          Yes, and then the users report bugs regressions in a version that is not supposed to have the patch that introduced it. This is literally what I am complaining about.

                                        2. 6

                                          Keep in mind that 1) this is literally only a problem on Linux distros or other third-parties repackaging applications and imposing their opinions on everybody 2) the actual context of this blogpost is that the author is mad at Python packages using Rust dylibs, it seems his sense of entitlement has not significantly improved since then.

                                          How is this relevant to static linking and the discussion about its security issues?

                                          1. 3

                                            Because it’s the reason this discussion continues to exist.

                                            1. 3

                                              So in summary people are still angry about cryptography and Rust and so they keep posting roundabout takes on it and people get onto news aggregator sites to hawk their positions but not work on a solution? I’m really not sure how that’s productive for anyone.

                                              1. 1

                                                I publish static binaries for my applications. Now I have a third party who wants to redistribute my free work but wants me to change the way I write software so their use of my free work gets easier (for a debatable value of easier).

                                                Frankly I don’t see a problem I have to solve. My way works just fine on Windows.

                                                1. 1

                                                  At this point it’s up to all the parties to coordinate. It’s obvious that each of the different parties have different perspectives, desires, and annoyances. If you put your shoes in any of the various parties (application developers, distro maintainers, application users, distro users), and there’s plenty on this thread and the HN version of this link, then I think you can see the many angles of frustration. I don’t think getting angry on message boards is going to settle this debate for anyone, unless you’re just looking to vent, which I’d rather not see on lobste.rs and instead on chatrooms.

                                          2. 5

                                            This is only a problem on Linux. The fact that anybody can create a Linux distribution means that there are lot of systems that are largely similar and yet wholly incompatible with one another. Bazaar-style development has encouraged this pattern and, as such, we have a fragmentation of Linux that have just the tiniest little differences that make packaging an app near impossible to do in an universal fashion. Like it or not, cathedral-style systems do not suffer from this problem. You can count on the libc and loader to exist in a well known and understood location in FreeBSD, Windows, and MacOS. Sure, there are going to be differences in between major versions, but not so much as the difference between glibc and musl.

                                            Having your own packaging system then frees you, the application developer, from having to wait on the over 9,000 different Linux distributions to update their packages so that you can use a new shiny version of a dependency in your app. Furthermore, there are plenty of commercial, proprietary, software packages that don’t need to move at the same cadence as their deployed Linux distribution. The app might update their dependencies more frequently while in active development or less frequently if the business can’t justify the cost of upgrading the source code.

                                            I lay out that this situation is not unique to Linux, but rather, it exists because of Linux’s fragmentation… And secondarily as a result of the friction associated with walled-garden ecosystems like Apple.

                                          1. 6

                                            Can’t wait for Generics and I don’t think I am ever going back to a JVM based language!

                                            1. 2

                                              Maybe go will be worth another look after generics. I guess it should make implementing a sequence/stream API more feasible. Although I suspect the performance would suck, go probably can’t optimize the virtual function calls as much as a JIT can.
                                              Coming from a JVM background, and having recently written a CLI app with go, I found the experience extremely painful, and I don’t quite understand why one would give up a higher level language to work with go for non-trivial applications.
                                              Being able to easily build and cross compile native binaries is a great feature, especially for CLI’s, but if running a JVM isn’t a major constraint, I’d take any major JVM language over go.

                                              1. 2

                                                This kind of reflects my views about Go as well. I think once you are out of “simplicity” dogma, you quickly realize how messy the code gets with interface{} casts everywhere. I use generics on daily base! Even a basic cache requires generic support. I don’t want to litter my code with castings and ifs when there exists a decent solution to do all of the manual undertaking for you. That is what compilers were invented for rather than just generating plain code. You can obviously ignore them if you don’t need them; but I not having them is a big pain in the a**.

                                                1. 7

                                                  you quickly realize how messy the code gets with interface{} casts everywhere.

                                                  It should essentially never happen that you use interface{} in day-to-day code. If you’re having that experience, I can understand why you’d be frustrated! You’re essentially fighting the language.

                                                  interface{} and package reflect are tools of last resort that really only need to be used in library code.

                                              2. 1

                                                Two more releases, likely :)

                                                I’m curious to see how the generics will work out in practice. But I do look forward to having a sane assert.Equal().

                                              1. 3

                                                Goof stuff. There’s also https://github.com/VictoriaMetrics/metrics lib that I suggest to people who want to add metrics exposition to their Go apps but don’t want to include a lot of dependencies from the official Prometheus client library.

                                                1. 2

                                                  Strong -1 — everything written by that guy is sloppy and full of caveats. For example, the Histogram type in this package uses dynamic buckets, making any kind of aggregation — an average, a rate over any dimension, etc. — totally statistically invalid.

                                                1. 21

                                                  The article never mentions the, in my humble opinion, most important part of good logging practices and that is structured logging. Without it you end up with weird regexes or other hacks trying to parse your log messages.

                                                  1. 4

                                                    As a sibling post notes, if you use structured logging you’re mostly throwing away the idea that the entries must be easily parsable by a human. If that’s the case, and we’ll need a custom method of displaying the structured logs in a human friendly way, I believe we should forego plain text all together and gain the benefits of logging directly to binary.

                                                    1. 5

                                                      You can do human readable structured logging if you use key="value" formats inside text messages. Some people still prefer json, but there is a middle ground.

                                                      1. 2

                                                        If you need just key=value, that’s not really structured in my opinion.

                                                        1. 4

                                                          Why not?

                                                          1. 2

                                                            Because the amount of information added by this format would be infinitesimal over a line based logger with manual tokenization. The reason why you’d want a structured logger is to allow proper context to a message. Unless you’re working with simple cases, the structure that would offer such context is more than one level deep.

                                                            1. 3

                                                              Hmm, definitely not.

                                                              Structured logging is about decorating log events with just enough of a schema to make them machine parseable, so that searching, aggregating, filtering, etc. can more than a crapshoot. Deeply nested events significantly increase the complexity of that schema, and therefore the requirements of the consumer.

                                                              By default, structured logs should be flat key/value pairs. It gets you the benefits of richer parseability, without giving up the ability to grep.

                                                    2. 2

                                                      Excellent point. That’s become such second nature to me by now, that I forgot to even mention it!

                                                      1. 1

                                                        On top of that, structured logger if implemented properly, can often be faster and be operated at granular levels (like the other comments pointed out, sometimes you do want to on-fly turn on some logs at some locations, not all logs at all locations).

                                                        1. 1

                                                          I love structured logging, with one caveat: the raw messages emitted (let’s assume JSON) are harder for me to scan when tailing directly (which I usually only do locally as we have better log querying tools in the cloud), in contrast to a semi-structured simple key-value format. Do you all use a different format than JSON? Or a tool that transforms structured logs to something more friendly to humans, eg. with different log levels displayed in different appropriate colors, eg. JSON syntax characters diminished, for local tailing?

                                                          1. 5

                                                            At Joyent, we used the Bunyan format. Each line in the file was a separate JSON object with standard properties, some mandatory and some optional, and freeform additional properties. We shipped a tool, bunyan, that was capable of acting as a filter that would render different human readable views of the JSON. For example, you would often run something like:

                                                            tail -F $(svcs -L manatee) | bunyan -o short
                                                            

                                                            It also had some rudimentary filtering options. It also had a relatively novel mode that would, instead of reading from a file or standard input, use DTrace probes for different log levels to allow you to dynamically listen for DEBUG and TRACE events even when those were not ordinarily present in the log files. The DTrace mode could target a particular process, or even all processes on the system that emitted Bunyan logs.

                                                            1. 1

                                                              Hi, what were the required fields? Was it just a unique request ID? Thanks for sharing about bunyan. Even though it’s been out for a while I was unaware of it.

                                                            2. 5

                                                              Do you all use a different format than JSON? Or a tool that transforms structured logs to something more friendly to humans, eg. with different log levels displayed in different appropriate colors, eg. JSON syntax characters diminished, for local tailing?

                                                              We use JSON and the only tools I use are grep and jq. And although I am pretty much still a novice with these two, I found that with the power of shell piping I can do almost anything I want. Sometimes I reach for the Kibana web interface, get seriously confused and then go back to the command line to figure out how to do it there.

                                                              I wrote a simple tutorial for the process, just a couple of weeks ago.

                                                              1. 1

                                                                If you rely on external tools to be able to make sense of your logs, why not go all the way, gain the speed and size benefits that binary logs would bring, and write your own log pager? I feel like the systemd folks had the right idea even when everyone was making fun of them.

                                                                1. 3

                                                                  I don’t think the average employer would be happy subsidizing an employee writing a log pager instead of implementing something that would bring a tangible result to the business. The potential money savings by using binary logs probably doesn’t outweigh the new subs/increased profits of churning out more features.

                                                                  1. 1

                                                                    To me that sounds like an excuse. The world is not made up of only software that is beholden to the all mighty shareholder.

                                                                    1. 1

                                                                      I mean, yes, if you’re developing something in your personal time, go bananas on what you implement.

                                                                      But I also know my manager would look at me funny and ask why I’m not just shoving everything into CloudWatch/<cloud logging service>

                                                                  2. 2

                                                                    I’m sure most problems with systemd journals are fixable, but they’ve left a very bad taste in my mouth for two main reasons: if stuff gets deleted from under them they apparently never recover (my services continue to say something like “journal was rotated” until I restart them), and inspecting journals is incredibly slow. I’m talking magnitudes slower than log files. This is at its worst (I often have time to make a cup of tea) when piping the output into grep or, as journalctl already does by default, less, which means every byte has to be formatted by journalctl and copied only to be skipped over by its recipient. But it’s still pretty bad (I have time to complain on IRC about the wait) when giving journalctl filters that reduce the final output down to a few thousand lines, which makes me suspect that there are other less fundamental issues.

                                                                    I should note that I’m using spinning disks and the logs I’m talking about are tens to hundreds of GB over a few months. I feel like that situation’s not abnormal.

                                                                    1. 1

                                                                      If you rely on external tools to be able to make sense of your logs, why not go all the way, gain the speed and size benefits that binary logs would bring, and write your own log pager?

                                                                      It’s hard to imagine a case at work where I could justify writing my own log pager.
                                                                      Here are some of the reasons I would avoid doing so:

                                                                      • Logs are an incidental detail to the application.
                                                                      • Logs are well understood; I can apply a logging library without issues.
                                                                      • My application isn’t a beautiful and unique snowflake. I should use the same logging mechanisms and libraries as our other applications unless I can justify doing something different.
                                                                      • JSON is boring, has a specification, substantial library support, tooling, etc.
                                                                      • Specifying, documenting, and testing a custom format is a lot of work.
                                                                      • Engineering time is limited; I try to focus my efforts on tasks that only I can complete.
                                                                      1. 2

                                                                        Logs are an incidental detail to the application.

                                                                        I think this is trivially disproved by observing that if the logs stop working for your service, that is (hopefully!) a page-able event.

                                                                        Logs are a cross-cutting concern, but as essential as any other piece of operational telemetry.

                                                                        1. 1

                                                                          Logs are a cross-cutting concern, but as essential as any other piece of operational telemetry.

                                                                          I rely heavily on logging for the services I support but the applications I wrote for work have only error reporting. They are used by a small audience and problems are rare; I might get a crash report every 18 months or so.

                                                                          1. 1

                                                                            Ah, yeah, I presume the context here is services.

                                                                    2. 1

                                                                      Agreed. jq is a really nice tool. It made the decision to transition to using JSON for logging very easy.

                                                                    3. 3

                                                                      Don’t use JSON, use logfmt.

                                                                      1. 1

                                                                        Yes! Logfmt is the good stuff. But it’s only semi-structured. Why not use JSON and a tool to transform to logfmt (with nested data elided probably) when needing to scan as a human?

                                                                        1. 1

                                                                          Logfmt is fully structured, it just doesn’t support nesting, which is an important feature! Structured logs should be flat.

                                                                    4. 1

                                                                      I’m surprised it wasn’t mentioned, but the larger advantage of passing a logger around to constructors is the ability to then have nested named loggers, such as

                                                                      Battery.ChargingStatus.FileReader: Failed to open file { file: "/tmp/battery charge", error: ... }
                                                                      Battery.ChargingStatus: Failed to access status logs, skipping report
                                                                      
                                                                    1. 10

                                                                      The solution is to use an object instance for your logger, the same as you should with a database, or other external dependencies, then use dependency injection to provide the logger to the code that generates logs.

                                                                      I know that’s the proper way to do things, but I hate having a logger instance past around to every single script and object. That’s what I used to do and I went back to a global instance, which can be initialised differently for test units or the app. Much simpler and good enough for my use. It makes the code a bit less modular since you have to make sure that this global logger has been initialised somewhere, but so much better than this extra dependency everywhere.

                                                                      In fact, I wonder if an object that is passed absolutely everywhere via dependency injection doesn’t in the end count as a global object anyway, and could be made one to simplify the code.

                                                                      1. 7

                                                                        I don’t think there can ever be a single “right” way to do this. The key is to know the caveats. If you’re willing to accept the extra cognitive load that comes with handling a global safely, as a trade-off against the extra overhead of passing objects around, I think that’s fine.

                                                                        I prefer the more explicit approach (especially on large projects, with many developers–many of whom will NOT be careful about global state), but of course YMMV.

                                                                        1. 2

                                                                          The way I like to look at it is that logging is an added behavior that we can separate from business logic. With this in mind, we can decorate the pieces of biz logic with logging at the entry point of the application. This way we can properly test biz logic without knowing anything about logging, and test logging without knowing anything about the biz logic. We can even inject faulty implementations to test logging when something goes wrong. And we can also decorate the dependencies of our dependencies… so turtles all the way down. An approach like this will inevitably push your codebase to be extremely modular. Too modular, to the point it can make people uncomfortable. I see this modularity as good thing, but not a lot of people share this same point of view… In any case, this is how I usually approach it. So +1 for dependency injection :-)

                                                                          Here’s a pseudo-code example:

                                                                          // ---
                                                                          
                                                                          interface Emailer {
                                                                            Send(EmailMessage)
                                                                          }
                                                                          
                                                                          // ---
                                                                          
                                                                          interface Logger {
                                                                            Trace(string)
                                                                          }
                                                                          
                                                                          // ---
                                                                          
                                                                          class StdoutLogger implements Logger {
                                                                            public Trace(message: string) {
                                                                              print(os.Stdout, message)
                                                                            }
                                                                          }
                                                                          
                                                                          // ---
                                                                          
                                                                          class SendGridEmailer implements Emailer {
                                                                            constructor(client: SendGridClient) {…}
                                                                          
                                                                            public Send(e: EmailMessage) {
                                                                              client.SendEmail(client.NewEmail(…))
                                                                            }
                                                                          }
                                                                          
                                                                          // ---
                                                                          
                                                                          class VerboseEmailer implements Emailer {
                                                                            constructor(logger: Logger, emailer: Emailer)
                                                                          
                                                                            public Send(e: EmailMessage) {
                                                                              logger.Trace("sending email")
                                                                              emailer.Send(e)
                                                                              logger.Trace("email sent")
                                                                            }
                                                                          }
                                                                          
                                                                          // ---
                                                                          
                                                                          main() {
                                                                            new VerboseEmailer(
                                                                              new StdoutLogger(),
                                                                              new SendGridEmailer(…)
                                                                            ).Send(new EmailMessage(…))
                                                                          }
                                                                          
                                                                        2. 2

                                                                          So don’t give it to everything. That means not everything can log. Which is great! Logging should be a considered and deliberate action, like any other important thing in your application.

                                                                          1. 1

                                                                            Sometimes there are good reasons for having a static variable in a program. This is one of them.

                                                                            1. 1

                                                                              Eh, nah. Few things are true process singletons, and loggers ain’t one of them.

                                                                            2. 1

                                                                              In fact, I wonder if an object that is passed absolutely everywhere via dependency injection doesn’t in the end count as a global object anyway, and could be made one to simplify the code.

                                                                              Keeping dependencies explicit ensures that each component is testable in isolation from others, and establishes a clear rule — the possible effects of a component are constrained by its inputs, i.e. no side effects — which helps enormously in the understanding (and therefore maintenance) of programs.

                                                                            1. 7

                                                                              I think this is the biggest advancement in Rust so far. I’ve been concerned about laying a finger on Rust for a long time because I have a lot of concern about Mozilla and it’s ethics. After the Brendan Eich ordeal it was difficult to rationalize any kind of investment in a company that behaved that way.

                                                                              Bringing Rust into the realm of the Software Foundation, I can say I’ll be following Rust with a renewed interest. Zig beat them to it, and I’d gladly consider a new project in Zig before I chose Rust, for obvious reasons, as I’m sure we’re all pretty sick of hearing about “what Rust can (kinda) do”, but all the same, these are some big names getting behind the project. That much can’t be ignored.

                                                                              2021 is shaping up to be a fast-moving year in PLs and PLT research as well.

                                                                              1. 15

                                                                                as I’m sure we’re all pretty sick of hearing about “what Rust can (kinda) do”

                                                                                At this point I hear more people saying this than I do people actually evangelizing Rust.

                                                                                1. 23

                                                                                  Brendan Eich did something that made it hard to believe he’d be fair and welcoming in a global project that extremely heavily depends on public goodwill and participation of volunteers.

                                                                                  (And he continues to make controversial, and frankly dangerous and stupid, public statements today. He denies that face masks work during a global pandemic, and actively discourages people from listening to public health experts, for example.)

                                                                                  His job was to be the face of the company. People freely chose not to associate with a company who chose someone controversial as their face. Enough people made this free choice that it seemed wise to pick someone else.

                                                                                  I never understood why this was so terrible. What is the alternative? Force people to use products and services they don’t want to use? Forbid people from making purchasing decisions based on how they feel about employees at a company?

                                                                                  1. 12

                                                                                    Enough people made this free choice that it seemed wise to pick someone else.

                                                                                    I never understood why this was so terrible.

                                                                                    TBH I assumed the bad behavior referred to was that they kept a shitbag like Eich around as long as they did.

                                                                                    1. 6

                                                                                      A diverse opinion being rejected in a group inherently portrays that group as exclusive in nature. Bubbling themselves in has alienated a lot of possibilities. Look at recent cuts Mozilla has to make, look at FF’s market share in the browser realm. I see W3 reporting FF as lower than 10% these days.

                                                                                      I don’t know about his opinions on these things, I’m not really trying to open a discussion about Eich, I’m not his follower, I am just presenting the novel idea that booting people for their personal opinions leads to failure and echo chambers and whatever else.

                                                                                      His JOB was co-founder. He CO-founded Mozilla. That’s different than being hired as CEO “here, go be the public figure, soak up those headline bullets and shut up on socials”.

                                                                                      Anyhow, I’m not a Mozilla history expert. I don’t think it’ll be relevant in 20 years, afaict it’s already dead.

                                                                                      Rust however, need not die, insofar as enough resources are dedicated to its longevity factor. Rust needs major reworking to be able to integrate new compiler core team, there’s major work needed to improve syntax issues, there’s HUGE work needed to do something about the compile times. I’ve seen users describe it as “not a better C, but a better C++” and I think that’s a decent designation for it for the time being. Still, without major pivoting and better resource and labor allocation, the project is in big trouble. I see people working on Rust who tweet 50-70+ a day. How productive can they really be???

                                                                                      It’s whatever. I really like the idea of a software foundation. It’s definitely going to be helpful to have diverse minds in diverse fields bouncing ideas around. It’s great.

                                                                                      1. 22

                                                                                        May I refer you to the Paradox of tolerance? Groups that want to maximize diversity must exclude those who are against diversity.

                                                                                        Eich gave public material support to Prop 8. He could have pretended he doesn’t support it, he could have “shut up on socials”, but he chose not to.

                                                                                        1. 16

                                                                                          I remember when he was debating this on Twitter. His response was that people not wanting to work with him because of his support of Prop 8 (which would make same-sex marriage illegal) was “fascism”.

                                                                                          Of course, he said this to people who were freely choosing to not associate with him based on their own opinions and his public statements…while he himself was supporting expanding government power to limit the kinds of associations consenting adults could participate in.

                                                                                          One of those is was way more “fascist” than the other.

                                                                                        2. 14

                                                                                          A diverse opinion

                                                                                          This characterization is both insufficient and inaccurate.

                                                                                          1. 12

                                                                                            His JOB was co-founder. He CO-founded Mozilla. That’s different than being hired as CEO “here, go be the public figure, soak up those headline bullets and shut up on socials”.

                                                                                            No one complained about him until they hired him to be the CEO. I didn’t even know his name before that and I bet a lot of other people are in the same boat. You seem really offended by something but you don’t seem to know what it is…

                                                                                            1. 7

                                                                                              Still, without major pivoting and better resource and labor allocation, the project is in big trouble

                                                                                              You would know better than I would, but this is honestly the first time I’ve ever heard anything other than “Rust is the language of the future and there’s no need to learn anything else ever.” I’m being only slightly facetious.

                                                                                              Seriously, though, from a mostly-outsider’s perspective, it seems like Rust is going nowhere but up and seems to be poised to take over the world. I suppose there’s a difference between Rust-the-language and Rust-the-project, but they’re pretty much identical to me.

                                                                                              1. 6

                                                                                                I see people working on Rust who tweet 50-70+ a day. How productive can they really be???

                                                                                                This is patently ridiculous as an argument.

                                                                                            2. 12

                                                                                              Mozilla did not own or control Rust at any point. The Rust project started out managed by Graydon Hoare in 2006, and Mozilla began financially supporting it in 2009 (link to the history). Mozilla did own the trademarks for the Rust and Cargo names and logos, which were controlled and licensed in an open manner, and protected only to the degree necessary to avoid implied official status or endorsement by the Rust project (link to the policy). Mozilla also paid for the salaries of developers who worked on Servo, for some period one of the two largest Rust projects (the other being the Rust compiler itself), as well as the salaries of some folks who worked on the Rust compiler. However, Mozilla did not exercise or influence the direction of the Rust language, and from an early period a majority of Rust’s contributors, including members of the Core Team and other Rust Teams, did not work for Mozilla.

                                                                                              1. 4

                                                                                                what Rust can (kinda) do

                                                                                                I’m curious what this bit refers to

                                                                                                1. 1

                                                                                                  This could refer to many different parts of a immature ecosystem like GUI programming

                                                                                              1. 8

                                                                                                For the example, its actually really useful to allow a program to listen on either a unix socket or a tcp socket just by a command line flag. The string arg is useful when passed through to users.

                                                                                                1. 9

                                                                                                  Yes. The fact that an address is a mostly-opaque string to programs allowed plan 9 to add support for ipv6, without touching userspace networking APIs at all.

                                                                                                  Often, the best thing your program can do is treat data the user passes it as an opaque blob, for as much of its travel through your code as possible.

                                                                                                  1. 3

                                                                                                    Yeah I would call that a form of polymorphism, just like file descriptors.

                                                                                                    Unix could have had different data types for disk files, pipes, and sockets, instead of using integers, but then you would lose the polymorphism. You wouldn’t be able to write select(), and shell couldn’t have redirects.

                                                                                                    It also same caveats because there are some things you can do on disk files that you can’t do on the others (seek()).

                                                                                                    Still, given the constraints of C, the untyped design is better.

                                                                                                    This relates to my comment about textual protocols in Unix: https://lobste.rs/s/vl9o4z/case_against_text_protocols#c_wsdhsm (which will hopefully appear on my blog in the near future)

                                                                                                    Basically text is “untyped” and therefore you get generic operations / polymorphism (copy, diff, merge, etc.).

                                                                                                    Types can inhibit composition.


                                                                                                    (I know somebody is going to bring up more advanced type systems. I still want to see someone write a useful operating system that way. I’m sure it can be done but it hasn’t so far AFAIK. Probably somewhat because of the expression problem – i.e. you want extensibility in both data types (files, sockets) and operations (read, write, seek) across a stable API boundary. )

                                                                                                    1. 1

                                                                                                      Basically text is “untyped” and therefore you get generic operations / polymorphism (copy, diff, merge, etc.).

                                                                                                      “Sir, this is an Arby’s.”

                                                                                                      (Go already has this problem. I do not see your point, nor your point about the expression problem, as the number of address types is enumerable, and set in stone..)

                                                                                                      1. 2

                                                                                                        The “challenge” is to show me an operating system WITH fine-grained types that does NOT have the O(M*N) explosion of code. That is, show me how it solves the polymorphism problem. Here’s a canonical situation of M data types and N operations, and you can generalize it with more in each dimension (M and N both get bigger):

                                                                                                        • The data types: the operating system has persistent disk files, IPC like pipes, and networking to remote machines.
                                                                                                        • Now for the “operations”
                                                                                                          • How do you simultaneously wait on events from files, IPC and networks? Like select() or inotify().
                                                                                                            • Ditto with signals and process exits – waitfd(), signalfd(), etc. (Unix fails here so Linux invented a few more mechanisms).
                                                                                                          • How do you do redirects? A shell can redirect from a file or a pipe. (Both Rob Pike and DJB complained about the lack of compositionality for sockets: https://cr.yp.to/tcpip/twofd.html. Pike doesn’t like the Berkeley socket API because it’s non-compositional. That’s why both Plan 9 and Go have something different AFAIK)
                                                                                                          • How do you copy from a disk to IPC, disk to network, IPC to network, etc. In Unix, cat can work in place of cp. netcat also sort of works, but I think Plan 9 does better because networking is more unified.

                                                                                                        The connection is:

                                                                                                        • When you have fine-grained types, you get O(M * N) problems, and the expression problem arises out of that. You have M data types and N operations.
                                                                                                        • If you have a single type like a file descriptor or a string, then you don’t have an O(M * N) problem. So you don’t have a composition problem, and the resulting explosion of code.

                                                                                                        In other words, types can inhibit composition. Again, not saying it can’t be done, but just that I haven’t seen an OS that addresses this.

                                                                                                        Plan 9 is more compositional precisely because it has FEWER types, not more. As mentioned, opaque strings are used as addresses.

                                                                                                        (Rich Hickey also has a great talk on this about the Java HTTP Request interface. It’s a type in the Java standard library that inhibits composition and generic operations. Link appreciated from anyone who knows what I’m talking about.)


                                                                                                        Kubernetes has a similar issue as far as I can tell:

                                                                                                        https://twitter.com/n3wscott/status/1355550715519885314

                                                                                                        I’m not familiar with the details, but I used the “predecessor” Borg for many years and it definitely has some composition problems. The Kubenetes ecosystem has a severe O(M * N) code explosion problem. Unix has much less of that, and Plan 9 probably has even less.

                                                                                                        Widely mocked diagram: https://twitter.com/QuinnyPig/status/1328689009275535360

                                                                                                        Claim: This is an O(M * N) code explosion due to lack of compositionality in Kubernetes.


                                                                                                        Also read Ken Thompson’s “sermonette” in his paper on the design of Unix shell:

                                                                                                        https://lobste.rs/s/asr9ud/unix_command_language_ken_thompson_1976#c_1phbzz

                                                                                                        A program is generally exponentially complicated by the number of notions that it invents for itself. To reduce this complication to a minimum, you have to make the number of notions zero or one, which are two numbers that can be raised to any power without disturbing this concept. Since you cannot achieve much with zero notions, it is my belief that you should base systems on a single notion.

                                                                                                        “Single notion” means ONE TYPE. Now this is taken to an extreme – Unix obviously does have both file descriptors and strings. But it uses those concepts for lots of things that would be modelled as separate types in more naive systems.


                                                                                                        Many familiar computing ‘concepts’ are missing from UNIX. Files have no records. There are no access methods. User programs contain no system buffers. There are no file types. These concepts fill a much-needed gap.

                                                                                                        Records are types. Unix lacks records. That’s a feature and not a bug. Records should be and ARE layered on top.

                                                                                                        In distributed systems, types should be layered on top of untyped byte streams.

                                                                                                        (This is going to be on the Oil blog. Many people have problems seeing this because it’s an issue of architecture and not code. It’s a systems design issue.)

                                                                                                        1. 1

                                                                                                          The problem is that different instances support different functionality. Seek works on some file descriptors, but not others. Ioctl is a horrific mess.

                                                                                                          Even COM was a better solution.

                                                                                                    2. 2

                                                                                                      without touching userspace networking APIs at all.

                                                                                                      This kind of statement sounds amazing, but has about 10 *’s next to it pointing out all the caveats. User space code still needed to change to account for new address inputs, otherwise who’s to say that “window.alert()” isn’t a valid address? You’re gonna pass arbitrary strings to that syscall?

                                                                                                      No. Create a constraint on the interface that ensures the caller isn’t an idiot.

                                                                                                      1. 4

                                                                                                        who’s to say that “window.alert()” isn’t a valid address?

                                                                                                        “window.alert()” IS a valid address for a unix domain socket. Maybe other things too.

                                                                                                        1. 1

                                                                                                          Unfortunate random string choice. :)

                                                                                                        2. 3

                                                                                                          How much risk does that actually mitigate in practice, and how much toil does it create?

                                                                                                          It is not true that stronger type systems, or stronger constraints, are strictly better in all circumstances.

                                                                                                          1. 5

                                                                                                            and how much toil does it create?

                                                                                                            In the example, the senior programmer spent nearly a day on this. It could have been a type error.

                                                                                                            It is not true that stronger type systems, or stronger constraints, are strictly better in all circumstances.

                                                                                                            I am not a static types apologist, and prefer to write stuff in Lispy languages, which are often quite dynamic, but I will never concede that two parameters that have a dependency on each other, as in a pair such as (socket type, address), are best represented by two arbitrary strings instead of an enumeration carrying a constrained value. That’s an absurd thing to argue. You’ve created an infinitely large set of potential inputs and told the programmer “Don’t worry! We’ll tell you if you’re wrong when you run the program.” How silly, especially when that state space can be reduced significantly, and the usage checked by the compiler we’re already paying for.

                                                                                                            1. 2

                                                                                                              In the example, the senior programmer spent nearly a day on this. It could have been a type error.

                                                                                                              Yep, that’s definitely toil, and it definitely could have been prevented by the compiler. But what about the toil that those type system features would bring to every other aspect of programming in the language? How do you measure that, and how do you weigh it? And not in the abstract, either, but in the specific context of Go as it exists today. Honest questions. I don’t know! But I do know it’s important to do, if you want a true reckoning of the calculus.

                                                                                                              Of course, this could also have been caught and prevented by a unit test. Or an integration test. Or during code review. Or by using an architectural model that was a better fit to the problem. There are many tools available to software engineering teams to mitigate risks, and each tool carries its own costs and benefits as it is applied in different contexts. If the game is mitigating risk — which it is — then not everything needs to be solved at the language level.

                                                                                                              I will never concede that two parameters that have a dependency on each other, as in a pair such as (socket type, address), are best represented by two arbitrary strings instead of an enumeration carrying a constrained value.

                                                                                                              Sure, that sounds good to me, but it’s much more abstract than the specific claim made in the article, which is that

                                                                                                              net.Dial("tcp", "1.2.3.4:9090")
                                                                                                              

                                                                                                              should become

                                                                                                              net.Dial(net.TCPIPAddr{
                                                                                                                  IP:   net.IPv4(1, 2, 3, 4),
                                                                                                                  Port: 9090,
                                                                                                              })
                                                                                                              

                                                                                                              This would certainly convert a class of bugs that are currently runtime errors into compile time errors. But it would also make every use of net.Dial substantially more laborious. Is the benefit worth the cost? I don’t think the answer is obviously yes or no.

                                                                                                            2. 3

                                                                                                              I don’t think it’s possible to have a clear answer to this—but FWIW, my experience is that in ecosystems where unioned string literal types are deployed widely (notably in TypeScript), there is effectively no meaningful dissent about whether that solution is strictly better than naked-string-only typing, in every practical measurable dimension. It makes docs better, it makes tooling and autocomplete better, it seems to prevent bugs, people generally just seem to like using it, and there are no appreciable downsides in practice (compile time impact is negligible, no implications for generics, etc.).

                                                                                                              I understand that in Go this does not quite pass muster, because Go’s methodology for evaluating language features implicitly (and significantly) discounts what most other languages would call “ergonomic” improvements. Other ecosystems are willing to rely on intuition for deciding what sorts of day-to-day activities are worth improving, and in those ecosystems, it is harder to argue that we should not make better type amenities around the most important data type in the field of CS (i.e., the string) for things that people do all the time (e.g., pass in “stringly-typed” arguments to functions), especially when there are no significant downsides (e.g., compilation speed, design subtlety, etc.).

                                                                                                            3. 3

                                                                                                              You’re gonna pass arbitrary strings to that syscall?

                                                                                                              To the (userspace) connection server, which decides how to best reach the address specified, and returns an appropriate error if the caller provides garbage – yes. Why not? Having one place that does all the work shared across the system makes it easy to have a single, consistent, interface with one location to search for bugs.

                                                                                                              There are, of course, a few asterisks: for example, the dns resolver needs to be taught about how to handle AAAA records, 6in4 tunnels need to know how to encapsulate, etc – but the programs that need this knowlege are generally the programs that provide the userspace APIs.

                                                                                                              No. Create a constraint on the interface that ensures the caller isn’t an idiot.

                                                                                                              Opacity is the strictest possible constraint on an interface. You may do nothing with the data, other than pass it on to something else. If the caller may not do anything, then the caller will not do anything stupid.

                                                                                                              1. 1

                                                                                                                Opacity is the strictest possible constraint on an interface. You may do nothing with the data, other than pass it on to something else. If the caller may not do anything, then the caller will not do anything stupid.

                                                                                                                Ok? But it’s not at all opaque in these examples. The two parameters actually relate to each other….

                                                                                                                1. 1

                                                                                                                  Yes. That’s a poor choice in the Go API – it should have been a single opaque blob, instead of two dependent ones. Plan 9 does not make this mistake.

                                                                                                        1. 5

                                                                                                          SICP brings wonders. The rest I don’t know.

                                                                                                          SICP: https://github.com/sarabander/sicp

                                                                                                          1. 5

                                                                                                            SICP is vastly overrated.

                                                                                                            1. 4

                                                                                                              Ok, I’ll bite. Why do you say that?

                                                                                                              1. 1

                                                                                                                I understand it doesn’t resonate with everybody, which is why I included it in the section of books good for specific interests.

                                                                                                            1. 7

                                                                                                              I don’t want to 💩on the author’s writeup here, because it is a decent one. I’m using it to launch another public objection to Go Generics.

                                                                                                              A lot of proposals for and write ups about Go Generics seem to miss that there’s a very large group of Go users who object to Generics, and for good reason. It’s not because this group questions the efficacy of generics in solving very specific problems very well – objectors are generally well attuned to Generics’ utility. What’s objected to is the necessity of Generics. The question that we pose is do we need generics at all? Are the problems that Generics solve so important that Generics should pervade the language?

                                                                                                              From the author’s conclusion

                                                                                                              I was able to solve a problem in a way that was not previously possible.

                                                                                                              Being able to solve problems in new ways isn’t always valuable; it can even be counter-productive.

                                                                                                              1. 24

                                                                                                                Nothing is necessary except an assembler. Well, you don’t even need the assembler, you can just flip the bits yourself.

                                                                                                                Go has an expressiveness gap. It has some kind of big classes of algorithms that can’t be made into libraries in a useful way. Most people advocate just rewriting basically the same code over and over forever, which is kind of crazy and error-prone. Other people advocate code-generation tools with go generate, which is totally crazy and error-prone, even with the decent AST tools in the stdlib. Generics close the gap pretty well, they’re not insanely complex, and people have had decades to get used to them. If you don’t want to use them yourself, don’t use them, but accept that there are people for whom, say, the ability to just go get a red-black tree implementation that they can use with a datatype of their own choosing, without loss of type-safety or performance, will greatly improve the usefulness of the language.

                                                                                                                Plus, from a purely aesthetic standpoint, it always seemed criminal to me to have a language that has first-class functions, and lexical closure, but in which you can’t even write map because its type is inexpressible.

                                                                                                                1. 9

                                                                                                                  Go has an expressiveness gap.

                                                                                                                  That’s true. You’ve identified some of the costs. Can you identify some of the benefits, too?

                                                                                                                  1. 12

                                                                                                                    Easy: not having a feature protects you from bright idiots that would misuse it.

                                                                                                                    Honestly though, that’s the only argument I can make against generic. And it’s not even valid, because you could say this about almost any feature. It’s a fully general counter argument: give people hammers, some will whack each other’s heads instead of hitting nails.

                                                                                                                    Assuming basic competency of the users and assuming they were designed from the ground up, generics have practically no downsides. They provide huge benefits at almost no marginal cost. There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                                                    1. 2

                                                                                                                      They provide huge benefits at almost no marginal cost.

                                                                                                                      If this huge benefit is only one in a minor part of the project, or even, in a minority of projects, then it has to be balanced and thought through.

                                                                                                                      Right now, I don’t know many people that work Go daily, telling me that not having generics makes their day a pain.

                                                                                                                      Most of them told me that it’s sometimes painful, but that’s actually pretty rare.

                                                                                                                      There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                                                      Is the burden really on them? To me the it is on the program writer.

                                                                                                                      1. 8

                                                                                                                        There’s likely a survivorship bias going on there.

                                                                                                                        I used Go as a programming language for my side projects for years. The thing that finally got me to give it up was the lack of generics. In writing PISC, the way I had approached it in Go ended up causing a lot of boilerplate for binding functions.

                                                                                                                        Go is something I’d happily write for pay, but I prefer expressiveness for my side projects now, as the amount of effort that goes into a side project is a big determining factor in how much I can do in one

                                                                                                                        1. 3

                                                                                                                          There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                                                          Is the burden really on them? To me the it is on the program writer.

                                                                                                                          Assuming we are a collaborative species (we mostly are, with lots of exceptions), then one of our goals should be minimizing total cost. Either because we want to spend our time doing something else, or because we want to program even more stuff.

                                                                                                                          For a moderately popular programming language, the users will far outnumber and outproduce the maintainers of the language themselves. At the same time, the languages maintainers’ work have a disproportionate impact on everyone else. To such a ludicrous extent in fact that it might be worth spending months on a feature that would save users a few seconds per day. Like compilation speed.

                                                                                                                          Other stuff like generic will affect fewer users, but (i) it will affect them in a far bigger way than shaving off a few seconds of compilation time would have, and (ii) those particular users tend to be library writers, and as such they will have a significant impact on the rest of the community.

                                                                                                                          So yes, the burden really is on the language creators and compiler writers.


                                                                                                                          Note that the same reasoning applies when you write more mundane software, like a train reservation system. While there is rarely any monetary incentive to make that kind of thing not only rock solid, but fast and easy to work with, there is a moral imperative not to inflict misery upon your users.

                                                                                                                      2. 5

                                                                                                                        I haven’t used Go in anger but here are some benefits from not including generics.

                                                                                                                        • Generics are sometimes overused, e.g. many C++ libraries.
                                                                                                                        • The type system is simpler.
                                                                                                                        • The compiler is easier to implement and high quality error messages are easier to produce.
                                                                                                                        • The absence of generics encourages developers to use pre-existing data structures.
                                                                                                                      3. 2

                                                                                                                        If red-black trees and map were just built in to Go, wouldn’t that solve 90% of the problem, for all practical purposes?

                                                                                                                        What I really miss in Go is not generics, but something that solves the same problems as multiple dispatch and operator overloading.

                                                                                                                        1. 3

                                                                                                                          Sort of, but no. There’s too many data structures, and too many useful higher-order functions, to make them all part of the language. I was just throwing out examples, but literally just a red-black tree and map wouldn’t solve 90% of the problem. Maybe 2%. Everyone has their own needs, and Go is supposed to be a small language.

                                                                                                                          1. 1

                                                                                                                            Data structures and higher-order functions can already be implemented in Go, though, just not by using generics as part of the language.

                                                                                                                      4. 15

                                                                                                                        Technically Go does have generics, they just aren’t exposed to the end developer, except in the form of the builtin map and array types, and are only allowed for internal developers. So in a sense, Go does need generics and they already pervade the language.

                                                                                                                        I don’t personally have a horse in this race and don’t work with Go, but from a language-design perspective it does seem strange to limit user-developed code in such a way. I’d be curious what your thoughts on why this discrepancy is OK and why it shouldn’t be fixed by adding generics to the language.

                                                                                                                        1. 14

                                                                                                                          I don’t personally have a horse in this race and don’t work with Go, but from a language-design perspective it does seem strange to limit user-developed code in such a way.

                                                                                                                          Language design is all about limiting user defined code to reasonable subsets of what can be expressed. For a trivial example, why can’t I name my variable ‘int’? (In Myrddin, as a counterexample, var int : int is perfectly legal and well defined).

                                                                                                                          For a less trivial example, relatively few languages guarantee tail recursion – this also limits user developed code, and requires programmers to use loops instead of tail recursion or continuation passing style.

                                                                                                                          Adding generics adds a lot of corner cases to the type system, and increases the complexity of the language a good deal. I know. I implemented generics, type inference, and so on in Myrddin, and I’m sympathetic to leaving generics out (or, as you say, extremely limited) to put a cap on the complexity.

                                                                                                                          1. 3

                                                                                                                            I see only two legitimate reasons to limit a user’s capabilities:

                                                                                                                            1. Removing the limitation would make the implementer’s life harder.
                                                                                                                            2. Removing the limitation would allow the user to shoot themselves in the foot.

                                                                                                                            Limiting tail recursion falls squarely in (1). There is no way that guaranteeing tail recursion would cause users to shoot themselves in the foot. Generics is another matter, but I strongly suspect it is more about (1) than it is about (2).

                                                                                                                            Adding generics adds a lot of corner cases to the type system, and increases the complexity of the language a good deal.

                                                                                                                            This particular type system, perhaps. This particular language, maybe. I don’t know Go, I’ll take your word for it. Thing is, if Go’s designers had the… common sense not to omit generics from their upcoming language, they would have made a slightly different language, with far fewer corner cases they will inevitably suffer now that they’re adding it after the fact.

                                                                                                                            Besides, the complexity of a language is never a primary concern. The only complexity that matters is that of the programs written in that language. Now the complexity of a language does negatively impact the complexity of the programs that result from it, if only because language space is bigger. On the other hand, this complexity has the potential to pay for itself, and end up being a net win.

                                                                                                                            Take C++ for instance. Every single feature we add to it increases the complexity of the language, to almost unbearable levels. I hate this language. Yet, some of its features definitely pay for themselves. Range for for instance, while it slightly complicates the language, makes programs that use it significantly cleaner (although only locally). That particular feature definitely pays for itself. (we could discuss other examples, but this one has the advantage of being uncontroversial.)

                                                                                                                            As far as I can tell, generics tend to massively pay for themselves. Not only do they add flexibility in many cases, they often add type safety (not in C++, they don’t). See for instance this function:

                                                                                                                            foo : (a -> b) -> [a] -> [b]
                                                                                                                            

                                                                                                                            This function has two arguments (where a and be are unknown types): a function from a to b, and a list of a. It returns a list of b. From this alone, there is a lot we can tell about this function. The core idea here is that the body of the function cannot rely on the contents of generic types. This severely constraints what it can do, including the bugs it can have.

                                                                                                                            So, when we write let ys = foo f xs, here’s what we can expect before we even look at the source code:

                                                                                                                            • Assuming f is of type a->b, then xs is a list of a, and the result ys is a list of b.
                                                                                                                            • The elements of ys, if any, can only come from elements of xs.
                                                                                                                              • And they must have gone through f.
                                                                                                                              • Exactly once.
                                                                                                                            • The function f itself does not affect the number or order of elements in the result ys
                                                                                                                            • The elements of xs do not individually affect the number or order of elements in the result ys
                                                                                                                            • The only thing that affects the number or order of elements in the result ys is the size of xs (and the code of foo, of course).

                                                                                                                            This is quite unlike C++, or other template/monomorphisation approaches. Done right, generics have the opportunity to remove corner cases in practice. Any language designer deciding they’re not worth their while better have a damn good explanation. And in my opinion, the explanations offered for Go weren’t satisfactory.

                                                                                                                            1. 4

                                                                                                                              Complexity of a language is the primary concern!

                                                                                                                              Languages are tools to express ideas, but expressiveness is a secondary concern, in the same way that the computer is the secondary audience. Humans are the primary audience of a computer program, and coherence is the primary concern to optimize for.

                                                                                                                              Literary authors don’t generally invent new spoken languages because they’re dissatisfied with the expressive capability of their own. Artful literature is that which leverages the constraints of it’s language.

                                                                                                                              1. 4

                                                                                                                                Literary authors don’t generally invent new spoken languages because they’re dissatisfied with the expressive capability of their own. Artful literature is that which leverages the constraints of it’s language.

                                                                                                                                Eh, I have to disagree here. Literary authors try to stretch and cross the boundaries the of their spoken languages all the time, specifically because they search ways to express things that where not yet expressed before. To give some uncontroversial examples, Shakespeare invented 1700 new words and Tolkien invented not one, but a couple of whole new languages.

                                                                                                                                I am but a very low level amateur writer, but I can tell you: the struggle with the tool to express your ideas is as real with spoken languages as it is with programming languages. It is an approach from another direction, but the results from spoken languages turn out to be as imperfect as those from programming ones.

                                                                                                                                1. 1

                                                                                                                                  I’d argue that constrained writing is more common, if nothing else than showing ones mastery of a shared language is more impressive than adding unknown elements.

                                                                                                                                  Tolkien’s Elvish languages, while impressively complete, are simply used as flavor to the main story. The entire narrative instead leans heavily on tropes and language patterns from older (proto-English) tales.

                                                                                                                                  1. 1

                                                                                                                                    Yes, you have a point. I mentioned Tolkien because that was the first writer that created a new language that I could come up with. But in the end, if you want to express an idea, then your audience must understand the language that you use, otherwise they will not get your message. So common language and tropes can help a lot.

                                                                                                                                    However, I think your mention of constrained writing is interesting. Because in a way, that Go does not have generics, is similar to the constraint that a sonnet must follow a particular scheme in form and content. It is perfectly possible to add generics to Go, the same way as it is very possible to slap another tercet at the end of a sonnet. Nothing is stopping you, really, Expect that then it would no longer be a sonnet. Is that a bad thing? I guess not. But still almost no-one does it.

                                                                                                                                    I’d say that the rules, or the constraints, are a form of communication too. If I read a sonnet, I know what to expect. If I read Go, I know what to expect. Because some things are ruled out, there can be more focus on what is expressed within the boundaries. As a reader you can still be amazed. And, the same as in Go, if what you want to express really does not fit in the rules of a sonnet, or if it is not worth the effort to try it, then you can use another form. Or another programming language.

                                                                                                                                  2. 1

                                                                                                                                    Your points don’t conflict with my points, and I agree with them.

                                                                                                                                  3. 2

                                                                                                                                    Can we agree that the goal of programming languages is to reduce costs?

                                                                                                                                    • Cost of writing the program.
                                                                                                                                    • Cost of errors that may occur.
                                                                                                                                    • Cost of correcting those errors.
                                                                                                                                    • Cost of modifying the program in the face of unanticipated new requirements.

                                                                                                                                    That kind of thing. Now we must ask what influences the costs. Now what about increased expressiveness?

                                                                                                                                    A more expressive language might be more complex (that’s bad), more error prone (that’s bad), and allow shorter programs (that’s good), or even clearer programs (that’s good). By only looking at the complexity of the language, you are ignoring many factors that often matter a whole lot more.

                                                                                                                                    Besides, that kind of reasoning quickly breaks down when you take it to its logical extreme. No one in their right mind would use the simplest language possible, which would be something like the Lambda Calculus, or even just the iota combinator. Good luck writing (or maintaining!) anything worth writing in those.

                                                                                                                                    Yes, generics makes a language more complex. No, that’s not a good enough argument. If it was, the best language would only use the iota combinator. And after working years in a number of languages (C, C++, OCaml, Ptython, Lua…), I can tell with high confidence that generics are worth their price several orders of magnitudes over.

                                                                                                                                    1. 2

                                                                                                                                      I agree with you that generics can be hugely net positive in the cost/benefit sense. But that’s a judgment that can only be made in the whole, taking into account the impact of the feature on the other dimensions of the language. And that’s true of all features.

                                                                                                                                      1. 1

                                                                                                                                        Just popping in here because I have minimal experience with go, but a decent amount of experience in languages with generics, and I’m wondering: if we set aside the implementation challenge, what are some examples of the “other dimensions” of the language which will be negatively impacted by adding generics? Are these unique to go, or general trade offs in languages with generics?

                                                                                                                                        To frame it in another way, maybe a naive take but I’ve been pretty surprised to see generics in go being rejected due to “complexity”. I agree that complexity ought to be weighed against utility but can we be a little more specific? Complexity of what specifically? In what way will writing, reading, compiling, running, or testing code become more complicated when my compiler supports generics. Is this complexity present even if my own code doesn’t use generics?

                                                                                                                                        And just a final comparison on language complexity. I remember when go was announced, the big ticket feature was its m:n threaded runtime and support for CSP-style programming. These runtimes aren’t trivial to implement, and certainly add “complexity” via segmented stacks. But the upside is the ability to ergonomically express certain kinds of computational processes that otherwise would require much more effort in a language without these primitives. Someone decided this tradeoff was worth it and I haven’t seen any popular backlash against it. This feature feels very analogous to generics in terms of tradeoffs which is why I’m so confused about the whole “complexity” take. And like, maybe another naive question, but wouldn’t generics be significantly less tricky to implement than m:n threads?

                                                                                                                                        1. 5

                                                                                                                                          It isn’t just implementation complexity of generics itself. It’s also sure to increase the complexity of source code itself, particularly in libraries. Maybe you don’t use generics in your code, but surely some library you use will use generics. In languages that have generics, I routinely come across libraries that are more difficult to understand because of their use of generics.

                                                                                                                                          The tricky part is that generics often provides some additional functionality that might not be plausible without it. This means the complexity isn’t just about generics itself, but rather, the designs and functionality encouraged by the very existence of generics. This also makes strict apples-to-apples comparisons difficult.

                                                                                                                                          At the end of the day, when I come across a library with lots of type parameters and generic interfaces, that almost always translates directly into spending more time understanding the library before I can use it, even for simple use cases. That to me is ultimately what leads me to say that “generics increases complexity.”

                                                                                                                                          1. 2

                                                                                                                                            what are some examples of the “other dimensions” of the language which will be negatively impacted by adding generics?

                                                                                                                                            From early golang blog posts I recall generics add substantial complexity to the garbage collector.

                                                                                                                                            The team have always been open about their position (that generics are not an early priority, and they will only add them if they can find a design that doesn’t compromise the language in ways they care about). There have been (numerous proposals rejected)[https://github.com/golang/go/issues?page=3&q=generics++is%3Aclosed+label%3AProposal] for varied reasons.

                                                                                                                                            Someone decided this tradeoff was worth it and I haven’t seen any popular backlash against it

                                                                                                                                            There’s no backlash against features in new languages, because there’s nobody to do the backlash.

                                                                                                                                            Go has already got a large community, and there’s no shortage of people who came to go because it was simple. For them, adding something complex to the language is frightening because they have invested substantial time in an ecosystem because of its simplicity. Time will tell whether those fears were well-founded.

                                                                                                                                      2. 1

                                                                                                                                        No, expressiveness is the only reason for languages to exist. As you say, humans are the primary audience. With enough brute force, any language can get any task done, but what we want is a language that aids the reader’s understanding. You do that by drawing attention to certain parts of the code and away from certain parts, so that the reader can follow the chain of logic that makes a given program or function tick, without getting distracted by irrelevant detail. A language that provides the range of tools to let an author achieve that kind of clarity is expressive.

                                                                                                                                        1. 2

                                                                                                                                          I think we are using “expressive” differently. Which is fair, it’s not really a well-defined term. But for me, expressiveness is basically a measure of the surface area of the language, the features and dimensions it offers to users to express different ideas, idioms, patterns, etc. Importantly, it’s also proportional to the number of things that it’s users have to learn in order to be fluent, and most of the time actually exponentially proportional, as emergent behaviors between interacting features are often non-obvious. This is a major cost of expressiveness, which IMO is systemically underestimated by PLT folks.

                                                                                                                                      3. 3

                                                                                                                                        I implemented generics. You’re trying to convince me that it’s worth implementing generics. Why?

                                                                                                                                        Besides, the complexity of a language is never a primary concern.

                                                                                                                                        I disagree. I think implementation matters.

                                                                                                                                    2. 2

                                                                                                                                      That’s an intersting observation; thanks for sharing it.

                                                                                                                                      they just aren’t exposed to the end developer

                                                                                                                                      I think this supports my point better than I’m able to. Language design is just as much about what is hidden from developers as what is exposed. That generics are hidden from end users is something I greatly appreciate about Go. So when I refer to generics, I’m referring to generics used by every day developers.

                                                                                                                                      I’d be curious what your thoughts on why this discrepancy is OK and why it shouldn’t be fixed by adding generics to the language.

                                                                                                                                      In my opinion the greatest signal that Go doesn’t need generics is the wonderfully immense corpus of code we have from the last decade – all written without generics. Much of it written with delight by developers who chose Go over other langauges for it’s pleasant simplicity and dearth of features.

                                                                                                                                      That is not to say that some of us offasionally could have written less code if generics were available. Particularly developers writing library or framework code that would be used by other developers. Those developers absolutely would have been aided by generics. They would have written less code; their projects may have cost less to initially develop. But for every library/framework developer there are five, ten, twenty (I can’t pretend to know) end user application developers who never had the cognitive load of genericized types foisted on them. And I think that is an advantage worth forgoing generics. I don’t think I’m particularly smart. Generics make code less readable to me. They impose immense cognitive load when you’re a new developer to a project. I think there are a lot of people like me. After years of Java and Scala development, Go to me is an absolute delight with its absence of generics.

                                                                                                                                      1. 6

                                                                                                                                        In my opinion the greatest signal that Go doesn’t need generics is the wonderfully immense corpus of code we have from the last decade

                                                                                                                                        I don’t have a ready example, but I’ve read that the standard library itself conspicuously jumped through hoops because of the lack of generics. I see it as a very strong sign (that’s an understatement) that the language has a dire, pervasive, need for generics. Worse, it could have been noticed even before the language went public.

                                                                                                                                        If you had the misfortune of working with bright incompetent architects astronauts who used generics as an opportunity to make an overly generic behemoth “just in case” instead of solving the real problem they had in front of them, well… sorry. Yet, I would hesitate to accuse the language’s semantics for the failings of its community.

                                                                                                                                    3. 7

                                                                                                                                      I don’t remember exact details, it was super long ago, but I once wanted to write an editor centered around using a nontrivial data structure (“table chain” or “string table” or whatever was the name). Also the editor had some display aspect structures (~cells of terminal). At some point I needed to be able to experiment with rapidly changing the type of the object stored both in the “cells” and “chains” of the editor (e.g. to see if adding styles etc. per character might make sense from architectural point of view). If you squint, those are both kind of “containers” for characters (haskeller would maybe say monads? dunno). I had to basically either manually change all the places where the original “character” type was used, or fall back to interface{} losing all benefits of static typing that I really needed. Notably this was long before type aliases which would have possibly allowed me to push a bit further, though it’s hard for me to recall now. But the pain and impossibility of rapid prototyping at this point was so big I didn’t see it possible to continue working on the project and abandoned it. Not sure if immediately then or some time later I realized that this is the rare moment where generics would be valuable in letting me explore designs I cannot realistically explore now.

                                                                                                                                      In other words, what others say: nontrivial/special-purpose “containers”. You don’t need them until you do.

                                                                                                                                      Until then I fully subscribed to “don’t need generics in Go” view. Since then I’m in “don’t need generics in Go; except when do”. And I had one more hobby project afterwards that I abandoned for exactly the same reason.

                                                                                                                                      And I am fearful and do lament that once they are introduced, we’ll probably see everyone around abusing them for a lot of unnecessary purposes, and that this will be a major change to the taste of the language. That makes me respect the fact that the Team are taking their time. But I do miss them since, and if the Team grudgingly accepts the current draft as passabke, this is such a high bar that it makes me extremely excited for what’s to come, that it will be one of the best ways how this compromise can be introduced. Given that most decisions in languages are some compromises.

                                                                                                                                      1. 6

                                                                                                                                        Yeah, Go is very much not a language for rapid prototyping. It expects you to come to the table with a design already in mind.

                                                                                                                                        1. 2

                                                                                                                                          Umm, what? Honestly not sure if you’re meaning this or being sarcastic (and if yes, don’t see the point). I prototyped quite a lot of things in Go no problem. I actually hold it as one of the preferred languages for rapid prototyping if I expect I might want to keep the result.

                                                                                                                                          1. 5

                                                                                                                                            I’m being totally serious. Go is chock full of stuff that makes typical rapid prototyping extremely difficult. A lack of a REPL. Compiler errors on unused variables. Verbose error handling. And so on. All of these things combine to make it harder to “design on the fly”, so to speak, which is what rapid prototyping frequently means.

                                                                                                                                            With that said, Go works great for prototyping in the “tracer bullet” methodology. That’s where your prototype is a complete and production quality thing, and the iteration happens at a higher level.

                                                                                                                                            1. 1

                                                                                                                                              Got it, thanks! This made me realize that I reach for different languages in different cases for prototyping. Not yet really sure why now. But I feel that sometimes the dynamic types of Lua make me explore faster, whereas sometimes static types of Go or Nim make me explore faster.

                                                                                                                                      2. 4

                                                                                                                                        I’m going to assume you’re arguing in good faith here, but as a lurker on the go-nuts mailing list, I’ve seen too many people say “I don’t think generics are necessary” or “I haven’t heard a good enough reason for the complexity of generics”. It’s worth pointing out the Go team has collected feedback. Ian Lance Taylor (one of the current proposal’s main authors) spends a large portion of time responding to emails/questions/objections.

                                                                                                                                        I read a comment from someone who was on the Kubernetes team that part of the complexity of the API (my understanding is they have a pseudo-type system inside) is based on the fact that proto-Kubernetes was written in Java and the differences between the type systems compounded with a lack of generics created lots of complexity. (NOTE I don’t remember who said this, and I am just some rando on the net, but that sounds like a decent example of the argument for generics. Yes, you can redesign everything to be more idiomatic, but sometimes there is a compelling need to do things like transfer a code base to a different language)

                                                                                                                                        1. 1

                                                                                                                                          Ouch, I was wondering why the Kubernetes API looks so painfully like Java and not like Go. TIL that’s because it was literally a dumb translation from Java. :/ As much as I’m a pro-generics-in-Go guy, I’m afraid that’s a bad case for an argument, as I strongly believe it is a really awful and unidiomatic API from Go perspective. Thus I by default suspect that if its authors had generics at their disposal, they’d still write it Java-style and not Go-style, and probably still complain that Go generics are different from Java generics (and generally that Go is not Java).

                                                                                                                                        2. 3

                                                                                                                                          I don’t know if the author’s example was a good one to demonstrate the value of generics, but a cursory look at the diff would suggest he didn’t really gain anything from it. I always thought a huge benefit of generics was it saved you 10s or even 100s of lines of code because you could write one generic function and have it work for multiple types. He ended up adding lines. Granted, the author said it was mostly from tests, but still there doesn’t seem to be any dramatic savings here.

                                                                                                                                          1. 3

                                                                                                                                            I recommend taking more than a cursory look. The value here is very much in the new library interface. In effect, the package provides generalize channels, and before the change, that generalization meant both a complicated interface, and losing compiler-enforced type safety.

                                                                                                                                        1. 27

                                                                                                                                          Very much no to all of this.

                                                                                                                                          “Reasoning about code” is as far as I understand it a proxy for building a correct mental model of the code, and then using that model to facilitate changes, and predictions about changes, effectively.

                                                                                                                                          This is a very real process, the most important thing to optimize for when writing software. It’s true that there are different approaches to it, which are each more or less effective in different contexts. But the task is to figure out what works and treat it as a priority — dismissing the entire idea as sophistry is naïve and self-defeating.

                                                                                                                                          1. 7

                                                                                                                                            Agreed. The author is kinda missing the point about when he dismissed “reasoning about code” due to the how to reason changing with time and being different between people.

                                                                                                                                            The only thing that matters is that you can reason about code. It doesn’t matter and I don’t care how you get there.

                                                                                                                                            1. 4

                                                                                                                                              There are also objectively easier-to-reason-about practices. Either they haven’t been exposed to it, or they’re just whining. The absolute first thing I thought of as an example: referential transparency, followed by pure functions.

                                                                                                                                          1. 6

                                                                                                                                            Although the author only sketches it, there is a left-hand path here; we can return to Free Software and force corporate consumers of the commons to comply with licenses which disgust them. As I have documented, some licenses are FSF-approved but not OSI-approved, and these licenses often are also not useful for the enumerated corporations.

                                                                                                                                            1. 3

                                                                                                                                              Thank you for the link to your answer on stackexchange. That was a very enlightening read!

                                                                                                                                              1. 3

                                                                                                                                                we can return to Free Software and force corporate consumers of the commons to comply with licenses which disgust them

                                                                                                                                                You literally can’t. The power dynamics don’t now, and won’t ever, allow it.

                                                                                                                                                1. 2

                                                                                                                                                  You didn’t read my second link; I provide evidence for my claim, including listing some companies which have made public statements or commitments to avoiding certain licenses even when those licenses cover essential software.

                                                                                                                                                  1. 2

                                                                                                                                                    …yes, which refutes your claim that you “can force corporate consumers . . . to comply with licenses which disgust them.” They’re opting out, not opting in; that’s not a win condition.

                                                                                                                                                    1. 2

                                                                                                                                                      It is very much a winning condition for the Free Software community. We don’t need for corporations to use our code, after all; it is not our problem if they must invent everything for themselves. We would hope, indeed, that if the community is large and robust enough, then corporations would be forced to be polite community members without special privileges, or else be completely outpaced by the combined momentum of folks using code which they’re not allowed to touch. Or, as I put it in a previous thread:

                                                                                                                                                      At scale, if the public commons is larger than any one corporation’s pool of coders, then this could prevent corporations from entering into public spaces which are broadly populated by people.

                                                                                                                                                      1. 1

                                                                                                                                                        [Someone not using our software] is very much a winning condition for the Free Software community.

                                                                                                                                                        Huh. That’s an… interesting, and rather tautological, way to define your terms. But thanks for the clarification.

                                                                                                                                                2. 1

                                                                                                                                                  WTFPL and Unlicense are disliked by entities that come with a legal department because the licenses don’t tick the right boxes for being a dependable tools (e.g. they lack a warranty disclaimer). I consider these anti-endorsements free legal advice because if the lawyers of a megacorp can’t make those licenses work for them (and it’s not just due to their business model or way of operating, like with Affero clauses) it’s likely that neither can I.

                                                                                                                                                  1. 3

                                                                                                                                                    I don’t think it’s true that you have the same legal needs and worries as large corporations. And this distinction is precisely what enables us to imagine that we have not completely run out of possibilities for using licensing alone to make progress with Free Software in society.

                                                                                                                                                    1. 2

                                                                                                                                                      I have different legal needs, so Affero clauses are fine for me personally. A license that is so blatantly US-centric as to potentially not be enforcable where I live is a legal worry for me just as it is for Google (does not live in Germany, but operates here). So, no Unlicense for me.

                                                                                                                                                      Disclosure: I work at Google, and my assessment of some of these decisions could be colored by off-the-cuff remarks made by open source team staff in random internal forums that may not be reflected in such detail in the public Google documentation.

                                                                                                                                                      However these arguments are also made by other parties, including the FSF, and they make sense to me: For example the argument that putting things in the public domain is not a thing in many countries (such as Germany), so there needs to be a clear alternative (Unlicense isn’t clear to me in that regard) and where the PD mechanism works there’s a risk that some default warranty pops up because you might not be able to disclaim copyright and at the same time disclaim warranty. That seems a bit contrived, but I’m not enough of a lawyer (as in, not at all) to rule that out completely, and it has been brought up.

                                                                                                                                                      So choosing Unlicense just because it’s off-limits for Google seems like a poor choice: Go AGPL or maybe EUPL, which have a similar anti-corporate effect while being written with some legal care (although EUPL has a bunch of uncertainties that make me reconsider every time I re-read it).

                                                                                                                                                      They also have the advantage of being fundamentally unfriendly towards “corporations that extract value from other people’s work without giving back” (which seems to be the main thrust of this entire exercise), while the rules that prohibit Unlicense, CC0 and other quasi-public domain arrangements at this time are entirely incidental and might be dropped if the legal risk assessment regarding public domain work in those corporations ever changes (for example if “PD with warranty disclaimer” gets support throughout the legal system in the US).

                                                                                                                                                    2. 1

                                                                                                                                                      The Unlicense has a warranty disclaimer.

                                                                                                                                                      1. 1

                                                                                                                                                        Well, the disclaimer thing was an example :-)

                                                                                                                                                        As for the Unlicense, it’s not quite clear to me what is supposed to happen under this license in a jurisdiction with copyright law that doesn’t permit dedicating works into the public domain. I guess the barebone license in the second paragraph is supposed to take over, but who knows?

                                                                                                                                                        In this case, I can also lean on the free legal advice by the FSF who proposes using the CC0 instead for ticking all the right boxes (while still considered not acceptable by Google’s lawyers, apparently).

                                                                                                                                                        0BSD seems to be the most accepted “I really don’t care what happens” license variant of all and it carefully avoids any explicit public domain dedication, making it work the same inside and outside the US.

                                                                                                                                                        1. 2

                                                                                                                                                          I was just pointing out what appeared to be an inaccuracy in your comment.

                                                                                                                                                          The questions surrounding the Unlicense are why I dual license my projects under the MIT and the Unlicense.

                                                                                                                                                  1. 8

                                                                                                                                                    I was initially worried that they dropped the “we will never break your old programs” promise but that is not the case.

                                                                                                                                                    https://github.com/golang/go/issues/40025

                                                                                                                                                    Obviously we cannot delete these three from io/ioutil. But we can move the implementations to io and leave wrappers behind. That will be easier for new users, and it lets more packages do general I/O without a dependency on os.

                                                                                                                                                    1. 1

                                                                                                                                                      Though I consider it to be a bit silly, it seems they plan to retain the package in a deprecated state (for forever?).

                                                                                                                                                      1. 1

                                                                                                                                                        I believe they will eventually remove it from newer versions as adoption gets wider spread.

                                                                                                                                                        1. 12

                                                                                                                                                          A future hypothetical “Go 2” might; the Go 1 compatibility promise is pretty clear that any program written in any version of Go 1 should always work with newer Go 1 versions (with a few exceptions that don’t apply here).

                                                                                                                                                          As I mentioned the other day, there are already a bunch of deprecated/frozen packages that aren’t removed for exactly this reason.

                                                                                                                                                          1. 3

                                                                                                                                                            Yeah, this is the problem with standard libraries. They have to stay on “semver v1” forever. You better get everything right on the first try, because everything has to be maintained forever.

                                                                                                                                                            I wish languages treated their standard library like every other package. 3rd party packages can exist in multiple versions, so semver-major changes are only a migration, and don’t have to break anyone.

                                                                                                                                                            1. 10

                                                                                                                                                              I wish languages treated their standard library like every other package. 3rd party packages can exist in multiple versions, so semver-major changes are only a migration, and don’t have to break anyone.

                                                                                                                                                              I am so glad Go doesn’t work this way. It means there aren’t “eras” of codebases that utilize totally different subsets of language features.

                                                                                                                                                              1. 10

                                                                                                                                                                Go absolutely has that. One prominent example is code written before the context package and code written after it.

                                                                                                                                                                And tooling wise, there are projects before Go modules and projects after it.

                                                                                                                                                                1. 3

                                                                                                                                                                  That’s missing the point. Code written before the context package will work when copied and pasted, patched, incrementally migrated, or otherwise mingled into code written after the context package.

                                                                                                                                                                  1. 7

                                                                                                                                                                    That’s a good difference to point out, sure. But I don’t think it means my comment has missed the point. There are still costs associated with shifting subsets of the language/std beyond the ability to copy & paste. Look at net/rpc. It is at best unwise and at worst unusable in the code that uses and relies on context cancellation. Because it is a package written in a different era of Go.

                                                                                                                                                                  2. 1

                                                                                                                                                                    Technically, yes; effectively, I don’t think so. Changes to the stdlib and tooling are infrequent, and almost always come as additions, rarely depreciations, and almost never iterations on existing idioms. Go code will still bit rot over time but I think it is categorically different than the e.g. Rust ecosystem.

                                                                                                                                                                    1. 2

                                                                                                                                                                      Rust code bitrots faster than Go code. Absolutely yes. But I don’t think there is a category difference. It’s a difference of degree. That’s a feature of Go that I appreciate, as the churn in the Rust ecosystem annoys me. And IMO, there is not nearly as much appreciation for the annoyance of churn in Rust’s culture as there is in Go’s culture. I would say the difference in philosophy here is probably a category change.

                                                                                                                                                                      But my point is that Go has eras too. Best not to thumb your nose at the idea. Maybe thumb your nose at the frequency or scale of changes instead. The point is that churn/eras still happens in the Go ecosystem. I’ve lived through it since Go 1.0.

                                                                                                                                                                  3. 3

                                                                                                                                                                    Just because you don’t name the “eras”, it doesn’t mean they don’t exist. You still have deprecated features. There will be “still using ioutil” codebases, and “moved off ioutil” codebases (and such implicit “era” for every other deprecated feature). There are always some laggards — that’s the reason why the no-breaking-changes promise exists.

                                                                                                                                                                  4. 4

                                                                                                                                                                    This isn’t a problem, it’s a benefit. I don’t want to have to rewrite my code to the flavor of the day if I want to use a new feature. Compatibility breaks in widely used libraries should come with a heavy burden of necessity.

                                                                                                                                                                    Even outside of the standard library, an attempt at compatible evolution is a big benefit.

                                                                                                                                                                    1. 1

                                                                                                                                                                      The reason that isn’t done is because the standard library contains a lot of glue, just like the language itself does. Major “non-standard” libraries with data structures that get used as API parts don’t want to make breaking changes for the same reason Python 3 was so painful. Everybody has to upgrade at once.

                                                                                                                                                                      This isn’t as much of a problem for Go interfaces, which are structurally typed, but it’s a serious problem for struct, which is nominally-typed. Plenty of Go functions, even in the standard library, accept struct parameters.

                                                                                                                                                                    2. 2

                                                                                                                                                                      Go (and Rust) seems to be rather amateurish in that regard.

                                                                                                                                                                      Having no actual removal policy is only one step above an “ignore compat” free-for-all.

                                                                                                                                                                      1. 6

                                                                                                                                                                        Wait, ‘maintain guaranteed compatibility’ is only one step above ‘ignore compatibility’? Sounds backwards to me.

                                                                                                                                                                        1. 0

                                                                                                                                                                          You think ‘ignore compatibility’ should be above ‘maintain guaranteed compatibility’?

                                                                                                                                                                          (I don’t really care, I think both are playing at roughly the same amateur level, i. e. we should do much better in the 21st century than that.)

                                                                                                                                                                          1. 1

                                                                                                                                                                            What do you propose? Which ecosystems out there do this right, in your opinion?

                                                                                                                                                                1. 49

                                                                                                                                                                  Good lord, how is it elegant to need to turn your code inside-out to accomplish the basic error handling available in pretty much every other comparable language from the last two decades? So much of Go is well-marketed Stockholm Syndrome.

                                                                                                                                                                  1. 16

                                                                                                                                                                    I don’t think that responding with a 404 if there are no rows in the database is that any language supports out of the box. Some frameworks do, and they all have code similar to this for it.

                                                                                                                                                                    1. 3

                                                                                                                                                                      And sadly so often error handling is often done in a poor manner in the name of abstraction, though really bad one that effectively boils down to ignore that things can go wrong, meaning that one ends up digging through many layers when they actually do go wrong.

                                                                                                                                                                      People eventually give up and copy paste StackOverflow[1] solutions in the hopes that one of them will work, even when the fix is more accidental and doesn’t fix the root cause.

                                                                                                                                                                      The pinnacle was once checking code that supposedly could not fail. The reason was that every statement was wrapped in a try with an empty catch.

                                                                                                                                                                      But back to the topic. Out of the box is all good and nice until you want to do something different which in my experience happens more often than one would think. People sometimes create workarounds. In the example of non existing rows, for example doing a count before fetch, so doing two queries instead of one, just to avoid cases where a no rows error would otherwise be thrown.

                                                                                                                                                                      Now i am certainly not against (good) abstractions or automation, but seeing people fighting against those in many instances makes me prefer systems where they can be easily be added and can easily be reasoned about, like in this example.

                                                                                                                                                                      [1] Nothing against StackOverflow, just blindly copy pasting things, one doesn’t even bother to understand.

                                                                                                                                                                    2. 10

                                                                                                                                                                      In what way is Go’s error handling turning my code inside out?

                                                                                                                                                                      1. 6

                                                                                                                                                                        Pike has set PLT back at least a decade or two.

                                                                                                                                                                        1. 7

                                                                                                                                                                          It is possible to improve the state of the art while also having a language like Go that is practical, compiles unusually fast and is designed specifically to solve what Google found problematic with their larger C++ projects.

                                                                                                                                                                          1. 8

                                                                                                                                                                            compiles unusually fast

                                                                                                                                                                            There is nothing unusual about it. It’s only C++ and Rust that are slow to compile. Pascal, OCaml, Zig and the upcoming Jai are decent. It’s not that Go is incredible, it’s that C++ is really terrible in this regard (not a single, but a lot of different language design decisions made it this way).

                                                                                                                                                                            1. 3

                                                                                                                                                                              For single files, I agree. But outright disallowing unused dependencies, and designing the language so that it can be parsed in a single pass, really helps for larger projects. I agree on Zig and maybe Pascal too, but in my experience, OCaml projects can be slow to compile.

                                                                                                                                                                              1. 2

                                                                                                                                                                                I’m enjoying tinkering with Zig but I do wonder how compile times will change as people do more and more sophisticated things with comptime.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  My impression from hanging out in #zig is that the stage 1 compiler is known to be slow and inefficient, and is intended as a stepping-stone to the stage 2 compiler, which is shaping up to be a lot faster and more efficient.

                                                                                                                                                                                  Also there’s the in-place binary patching that would allow for very fast incremental debug builds, if it pans out.

                                                                                                                                                                              2. 2

                                                                                                                                                                                Don’t forget D.

                                                                                                                                                                              3. 1

                                                                                                                                                                                My experience with Go is that it’s actually very slow to compile. A whole project clean build might be unusually fast, but it’s not so fast that the build takes an insignificant amount of time; it’s just better than many other languages. An incremental build, however, is slower than in most other languages I use; my C++ and C build/run/modify cycle is usually significantly faster than in Go, because its incremental builds are less precise.

                                                                                                                                                                                In Go, incremental builds are on the package level, not the source level. A package is recompiled when either a file in the same package changes, or when a package it depends on changes. This means, most of the time, that even small changes require recompiling quite a lot of code. Contrast with C, where most of the time I’m working on just a single source file, where a recompile means compiling a single file and re-linking.

                                                                                                                                                                                C’s compilation model is pretty bad and often results in unnecessary work, especially as it’s used in C++, but it means that you can just work on an implementation by just recompiling a single file every build.

                                                                                                                                                                                1. 1

                                                                                                                                                                                  I have not encountered many packages that take more than one second to compile. And the Go compiler typically parallelizes compilation at the package level, further improving things. I’m curious to see any counter examples, if you have them.

                                                                                                                                                                              4. 4

                                                                                                                                                                                I don’t remember anyone in POPL publishing a PLT ordering, partial or total. Could you show me according to what PLT has been set back a decade?

                                                                                                                                                                              5. -2

                                                                                                                                                                                Srsly, I was looking for simpler, and was disappointed by the false promise.

                                                                                                                                                                              1. 1

                                                                                                                                                                                Why would you use server-sent-events rather than a websocket?

                                                                                                                                                                                1. 4

                                                                                                                                                                                  It’s considerably simpler, and has built-in affordances for re-connection and event replay.

                                                                                                                                                                                1. 6

                                                                                                                                                                                  (Reading the proposal) I am not convinced by type lists as proposed, they seem un-moduloar.

                                                                                                                                                                                  Summary: The bounds on type parameters are “constraints”, which are an extension of existing interface types (which describe a set of methods that the object should have). This makes it easy to declare that values of a parameter type must have a certain method. However, we may also want to use operators / multimethods on those values, and interface types are not expressive enough to specify this. The solution in the Generics proposal is type lists, basically you can define a constraint with a hardcoded list of allowed “underlying types” for your values (in addition to interface constraints).

                                                                                                                                                                                  For example to say “x < y should be supported on values of my type parameter”, the proposal suggests to define the following constraint:

                                                                                                                                                                                  // Ordered is a type constraint that matches any ordered type.
                                                                                                                                                                                  // An ordered type is one that supports the <, <=, >, and >= operators.
                                                                                                                                                                                  type Ordered interface {
                                                                                                                                                                                  	type int, int8, int16, int32, int64,
                                                                                                                                                                                  		uint, uint8, uint16, uint32, uint64, uintptr,
                                                                                                                                                                                  		float32, float64,
                                                                                                                                                                                  		string
                                                                                                                                                                                  }
                                                                                                                                                                                  

                                                                                                                                                                                  I think this is un-modular. This means the standard library will contain a single, authoritative list of all types that can use the comparison operator in generic code. Maybe it’s okay for numeric comparison, but in general this approach sounds like it will add a lot of friction. You want to allow library authors to introduce concepts, and then library users to introduce their own user-defined types satisfying these concepts (as interface types allow for methods as opposed to operators); but this design forces library authors to decide in advance of a fixed list of admitted types.

                                                                                                                                                                                  If you compare with Haskell type classes or Rust traits or Scala implicits, in those languages it is easy to express that comparison operators for a type should be available. In Haskell for example Ord a is a constraint that basically requires that an operator (<) :: a -> a -> Bool is available, and usage of x < y under an explicit Ord a constraint will correctly infer that this is the operator we mean. In other words, a constraint on a type a come with operations that mention a, in more flexible ways than just “I want values at a to have this method”. Ord a is an example of constraint allowing to use a binary operators, but for example Default a is a constraint with that provides a default value of type a (default :: a), so here there is no value of a to call the operation on, a is the output of the overloaded operation default.

                                                                                                                                                                                  1. 0

                                                                                                                                                                                    You want to allow library authors to introduce concepts, and then library users to introduce their own user-defined types satisfying these concepts (as interface types allow for methods as opposed to operators); but this design forces library authors to decide in advance of a fixed list of admitted types.

                                                                                                                                                                                    I don’t think this is true - the example you give is special, because the standard library knows only those types can be comparable, since there’s no operator overloading.

                                                                                                                                                                                    Can you give an example of your issue without using that special case? My understanding is that library authors can define a type constraint that uses an interface, and downstream can then just implement that interface.

                                                                                                                                                                                    Eg.

                                                                                                                                                                                    interface Comparable {
                                                                                                                                                                                        CompareTo(other Comparable) int
                                                                                                                                                                                    }
                                                                                                                                                                                    
                                                                                                                                                                                    func SomeAlgo[T Comparable](input []T) {
                                                                                                                                                                                        ..
                                                                                                                                                                                    }
                                                                                                                                                                                    
                                                                                                                                                                                    1. 2

                                                                                                                                                                                      As I said: it depends on how your concepts can be formulated. If they fit the mold of interfaces (they are about methods that you can call on the values of the parametrized type), then this works well. But when they don’t, it does not. Binary methods / multi-parameter functions like “compare” are not easily presented as interfaces.

                                                                                                                                                                                      In your example, nothing tells you that the two elements to be compared have to be of the same type, while you get this assurance for free with a signature such as (<) :: a -> a -> Bool (it says that the operator < takes two a and returns a Bool). In practice the CompareTo function for a user-defined type will do a type check and fail if the argument does not have the same type, but you can still easily mix things up, pass an argument of the wrong type to a CompareTo method, and get no warning that things are wrong. This is precisely the sort of imprecision that people are trying to solve by adding genericcs in the first place.

                                                                                                                                                                                      1. 1

                                                                                                                                                                                        As I said: it depends on how your concepts can be formulated. If they fit the mold of interfaces (they are about methods that you can call on the values of the parametrized type), then this works well. But when they don’t, it does not.

                                                                                                                                                                                        “Methods that you can call on values” is the only way to express concepts in Go. So I don’t think this is really a restriction. Or, at least, not a new restriction. I understand why some people would feel encumbered by this lack of expressivity — I do too, sometimes — but it’s an important part of what keeps Go so simple.

                                                                                                                                                                                        1. 2

                                                                                                                                                                                          This restriction was apparently problematic enough to the authors of the proposal that it’s their third version trying to devise mechanisms to avoid it (two previous versions used “contracts” presented under different forms). In my top comment, I am making the point that the mechanism they currently propose to work-around the restriction, namely “type lists”, is un-modular and may turn out to cause friction in practice, because it assumes that the person who defines a generic bound knows about all potential users of the bound (technically, this is often called a “closed-world assumptIon”).

                                                                                                                                                                                          (The previous reply made the point that the example is specific to a hardcoded operator of the standard library, so this is fine. I think that if the mechanism was only meant to apply in a couple hardcoded cases, it could be hidden underneath a few predeclared “primitive contracts” (such as comparable, compiler-defined without showing the definition) and not exposed to users. If it is exposed to users, it is probably because there are intended uses in user-libraries.)

                                                                                                                                                                                          1. 2

                                                                                                                                                                                            I believe type lists only exist to allow generic functions to specify which built-in language syntactic forms are allowed on the generic types. The reason why comparable is compiler defined is because it’s impossible to actually define with type lists. Equality (== and !=) is indeed an open set of underlying types (namely, some struct types). Every other built-in bit of syntax does follow the closed-world assumption: there is a finite list of underlying types that support <, for example. I cannot, at least, think of any reason to use type lists except to allow support for things like indexing, range, operators, etc.

                                                                                                                                                                                            I think the reason why they did not define a set of built in primitive contracts is because there’s something of an explosion of possibilities and required names. Here’s a table I made of the different ways a type can be used and what underlying types support them (that I make no claims about being fully correct or exhaustive, it was just a quick attempt):

                                                                                                                                                                                            operation              underlying types
                                                                                                                                                                                            --------------         ----------------
                                                                                                                                                                                            <, >, <=, >=           [u]int?, string, float?
                                                                                                                                                                                            ==, !=                 [u]int?, string, float?, complex?, some structs
                                                                                                                                                                                            +                      [u]int?, string, float?, complex?
                                                                                                                                                                                            -, *, /                [u]int?, float?, complex?
                                                                                                                                                                                            |, ^, %, &, &^         [u]int? 
                                                                                                                                                                                            <<, >>                 [u]int? but right hand side must be uint?
                                                                                                                                                                                            &&, ||, !              bool
                                                                                                                                                                                            x <- y                 chan T, chan<- T
                                                                                                                                                                                            <-x                    chan T, <-chan T
                                                                                                                                                                                            x[n]                   [n]T, *[n]T, []T, map[K]V, string
                                                                                                                                                                                            x[a:b]                 [n]T, *[n]T, []T, string
                                                                                                                                                                                            x[a:b:c]               [n]T, *[n]T, []T
                                                                                                                                                                                            range                  [n]T, *[n]T, []T, map[K]V, string, chan T, <-chan T
                                                                                                                                                                                            T{}                    [n]T, []T, map[K]V, structs
                                                                                                                                                                                            
                                                                                                                                                                                            builtins               underlying types
                                                                                                                                                                                            ---------------        ----------------
                                                                                                                                                                                            len(x)                 [n]T, *[n]T, []T, map[K]V, chan T, <-chan T, chan<- T, string
                                                                                                                                                                                            cap(x)                 [n]T, *[n]T, []T, chan T, <-chan T, chan<- T
                                                                                                                                                                                            make(T)                map[K]V, chan T, <-chan T, chan<- T
                                                                                                                                                                                            make(T, n)             map[K]V, chan T, <-chan T, chan<- T, []T
                                                                                                                                                                                            make(T, n, m)          []T
                                                                                                                                                                                            delete(m, k)           map[K]V
                                                                                                                                                                                            
                                                                                                                                                                                            other special cases    why
                                                                                                                                                                                            -------------------    ---
                                                                                                                                                                                            append(t []T, ts ...T) but if T is byte, then ts can be a string
                                                                                                                                                                                            copy(d, s []T)         but if T is byte, then s can be a string
                                                                                                                                                                                            complex(a, b)          float? and outputs complex?
                                                                                                                                                                                            real(a), imag(a)       complex? and outputs float?
                                                                                                                                                                                            

                                                                                                                                                                                            I hope that demonstrates some of the complexity of attempting to come up with a taxonomy of the allowed syntactic forms and why it might be reasonable to just leave that up to the users, especially when it is, indeed, a closed world except for equality. If I recall, the idea of defining builtin contracts was brought up many times during the design iteration, and it always failed when working out the specific details.

                                                                                                                                                                                            All that said, I find type lists to be particularly inelegant, and my least favorite aspect of the design. I hope something else is found to solve the problem: The desire to explicitly specify what syntax you are allowed to use on generic types so that you cannot silently break users of your generic function during a refactoring. For example, changing a usage of T{} to make(T) isn’t always valid for every T.


                                                                                                                                                                                            Also, as you noted, the example above that gave a non-type safe compare interface and function using it. The type safe version would be

                                                                                                                                                                                            type Comparable[T any] interface {
                                                                                                                                                                                            	CompareTo(T) int
                                                                                                                                                                                            }
                                                                                                                                                                                            
                                                                                                                                                                                            func SomeAlgo[T Comparable[T]](input []T) {
                                                                                                                                                                                            	..
                                                                                                                                                                                            }
                                                                                                                                                                                            

                                                                                                                                                                                            which requires methods of signature like

                                                                                                                                                                                            func (m MyType) CompareTo(n MyType) int { ... }
                                                                                                                                                                                            

                                                                                                                                                                                            to satisfy the interface.

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              Great reply, thanks!

                                                                                                                                                                                              1. 1

                                                                                                                                                                                                Your example defines an interface ComparableWith[T] of things that can be compared with T (I made a slight renaming to clarify my question). Is it possible in Go to express the interface of “being comparable to oneself”? I can always use [T ComparableWith[T]] to express the same bound, but it would be nice to be able to write

                                                                                                                                                                                                type Comparable interface {
                                                                                                                                                                                                  CompareTo(Self) int
                                                                                                                                                                                                }
                                                                                                                                                                                                
                                                                                                                                                                                                fun SomeAlgo[T Comparable](input []T) { ... }
                                                                                                                                                                                                

                                                                                                                                                                                                I’m not aware of a way to express those “self types” in Go (and I can’t find them in the specification), but I’m not a Go expert so I thought I would ask.

                                                                                                                                                                                                1. 1

                                                                                                                                                                                                  There is no “self type” currently in the language and I don’t believe anything like that exists in the current proposal, so I do not believe so.