1. 1

    Flagged for press release and advertising, and it isn’t even tagged properly.

    1. 1

      Fair enough. I did think it was interesting timing given the prior discussions around open source monetization with Redis.

      What would the correct tags have been?

      1. 1

        The nearest I could argue for would be release (since it’s a new program), but even then it’s a business thing and not a “hey, go download new software with changes X, Y, Z” thing.

    1. [Comment removed by author]

      1. 1

        out of curiosity, what do you mean?

        1. [Comment removed by author]

          1. 5

            These are great points! Part of why I changed our interview process back then (I’m no longer with Space Monkey) was to actually try and address a lot of these issues.

            So, yeah, gotcha trivia questions are terrible! That’s why we stopped whiteboard interviews entirely. They aren’t representative. What we wanted was a way of evaluating a candidate in a less stressful environment (on their own computer, with docs, their IDE, without people staring over their shoulder) doing something more close to day-to-day work.

            We picked homework problems as kind of the least terrible way of doing this. The downside of homework problems is the time issue. We replaced whiteboard problems (and time you’d have to spend to come in to the office for them), so this wasn’t in addition to other interviewing. That said, perhaps the homework problem takes longer. Hopefully the homework problem doesn’t take longer than the time it would have taken to come in and do whiteboard problems for a few hours, but still. Because of that, we decided that perhaps it would be better if we paid people to do those homework problems. The paying people is to try and be cognizant of people’s time and make up for it.

            I think interviewing is a two-way street so we gave candidates the option of coming to work with us for a day or two on real world tickets. They didn’t have to do this, but if they wanted to we were more than happy to pay them for a short term contract. It wasn’t a requirement, but just an option. I’m all about giving people options. In no case did we ever make someone do a whiteboard problem AND a homework problem AND a short term contract. It was always strictly zero or one of those options. Never more than one.

            Last, if a candidate had a body of code samples they wanted to supply us instead that were sufficient, we were more than happy to use those instead of a homework problem. That’s why I said zero or one before.

            Incidentally this is still the interview process I use at storj.io. We pay more at Storj for homework problems than I was able to at Space Monkey, which is good, but I’m always trying to improve it to make it better.

            Does any of this background make it seem less objectionable? I’m 100% on board with using published papers and open source software and references as the interview if those things are all available, but when that’s not available? By having a homework problem, someone who doesn’t have a catalog of existing work has a real shot at proving themselves to be a very capable candidate in a way that lines up with what work actually is.

      1. 4

        Storj Labs is (job description). We recently hired Ben Golub as CEO, which has been an injection of rocket fuel in terms of upcoming partnerships. We started rebuilding our platform in Go as of April of this year. Tons of interesting work in distributed storage! We’re very remote friendly.

          1. 1

            Can you imagine someone writing a blog post like “farewell, ball-peen hammers! I will be using framing hammers from now on!”

            1. 2

              nice, i wrote a suite of tools that interoperate with this exact file format description a few years ago. i think the csv to tsv conversions might replace tabs with spaces instead of erroring but otherwise this is exactly that

              https://github.com/jtolds/tsv-tools

              1. 2

                Very nice! I added a link to it from my doc

              1. 4

                Any example of a situation in which creating commits like this helps?

                1. 3

                  Let’s say you have some complicated history pattern with merges and so on. Lots of different developers doing lots of different things. After a bunch of merges and merge conflict resolutions you have a history of sorts, but you want to clean it up. This allows you to make a single commit where the end result is the tree matches the complex history you’d like to throw away.

                  Very useful for keeping dev history clean.

                  1. 1

                    Ah, that makes sense. So like a squashed merge, but without the merge. I guess it’s what git merge --squash --strategy=theirs would do if that merge strategy existed.

                    1. 1

                      Well, now that I think about it, wouldn’t saying –strategy=theirs be specifying just how conflicts are handled? My tool is saying, forget about conflicts, merging, everything, take the entire tree from the other commit wholesale. Don’t even try and merge things together.

                      1. 1

                        No, that’s what --strategy=recursive -X theirs does. The existing “ours” strategy just throws away the other commit and takes the tree from the current one. A fictional “theirs” strategy would do the same with the other tree.

                        Merge strategies and their options are pretty confusing.

                    2. 1

                      Wait… you use it to delete history? But… having that history around is the reason I use git?

                      1. 1

                        The last thing I want to do when fighting a production fire at 3am is be sorting through 12 merges of commits that look like:

                        • add feature
                        • whoops
                        • small fix
                        • review comments
                        • doh maybe this time.

                        squash that crap together! What commit broke the build is infinitely harder to figure out when the problem is in some chain of merges titled “whoops”

                        I decidedly prefer having my git history serve as a neatly curated form of documentation about the evolution of the codebase, not chaos of immutable trial and error

                        1. 2

                          I constantly bring this up in pull requests when I see shitty commit histories like that. Squash your damn commits! If you’ve already pushed a branch, create a new one with a new name, pick your commits on top of it, rebase -i and squash them into succinct relevant feature sets (or try to get as close as you can).

                          I realize this is once that’s already gone and it’s too late (history with a ton of “squishme: interum commit” bullshit in there) and that’s the purpose of tools like yours, but teaching people good code hygiene is pretty important too. :-P

                          1. 1

                            So I agree with you on this approach, but I think I’m still not grasping what your tool accomplishes. Couldn’t the situation you’re outlining here be accomplished by squashing?

                            1. 1

                              Yeah that last comment was really more of a discussion about why you might want to clean up git history. That’s a poor example for this tool.

                              This tool is useful when there’s multiple merges along two divergent branches of history and you want to make a commit that essentially contains the entire diff from your commit down to the merge-base of another commit combined with the diff from the merge-base back up to that other commit.

                              1. 1

                                Hmm, I guess I just can’t picture in what kind of situation that would happen. Could you explain the example chronologically?

                                1. 1

                                  I think @jtolds is on significantly more complicated code bases than I’ve worked on. There was an earlier post about Octopus commits:

                                  https://www.destroyallsoftware.com/blog/2017/the-biggest-and-weirdest-commits-in-linux-kernel-git-history

                                  and here is a visual for what that would look like:

                                  https://imgur.com/gallery/oiWeZmm

                    1. 3

                      A totally useful comment from Reddit: git commit-tree refspec^{tree} -p HEAD was exactly what I’ve always been looking for and does 95% of what my tool does.

                      https://www.reddit.com/r/git/comments/8v3pjg/comment/e1kkmhm

                      1. 10

                        Isn’t this the same as git reset <commit>; git commit … ?

                        1. 14

                          Oh my god you’re right. Edit: none of this comment is sarcasm! genuinely dumbfounded

                          git checkout branch1
                          git treesame-commit branch2 # a complex program
                          

                          is the same as

                          git checkout branch1
                          branch1ref=$(git log --format=%H -1)
                          git reset --hard branch2
                          git reset $branch1ref
                          git add .
                          git commit -m "treesame-commit" # no complex program
                          

                          and this whole time I believed there was no way to do this!

                          In my literally 7 years of using my own custom tool that writes git commit objects by hand, how have I never realized that? :headdesk:

                          Of course, to make the latter really solid you’d also want a git clean -dffx, git submodule sync, git submodule update --init --recursive, etc. In some sense, it’s hard to beat the concreteness of writing out the exact tree hash you want into a commit object, so frankly I’ll probably still keep using my tool but I do feel a bit sheepish.

                          Another person on reddit pointed out git read-tree and git checkout-index which also can solve the problem.

                          1. 3

                            For the 1st half of this comment, I thought you were being sarcastic!

                            1. 1

                              Nope! Just dumbfounded by my own stupidity. I agree though, I don’t know how to convey the dumbfoundedness without sounding sarcastic. Internetting is hard.

                            2. 3

                              git rev-parse $refdescriptor is a shorter more descriptive way to get the ref by the way

                          1. 5

                            oh this is weird to find a post about me at my internet watering hole.

                            well shucks

                            1. 11

                              Lots of other things to comment on but I firmly think interfaces-as-structural-types is one of Go’s greatest strengths. To put it in the “bad” group is upsetting and kind of made me discount the rest of the article.

                              1. 6

                                I think it’s a philosophical difference:

                                Some developers write code to document to coworkers what they are doing, and some developers just want things to compile with the least effort possible.

                                1. 2

                                  It’s not a matter of least effort. Go is one of the first languages I know of that was primarily designed for large teams of engineers instead of individuals. Heavy focus on compile time, gofmt, not allowing compilation with unused variables, etc, all directly stem from this approach. Structural typing specifically reduces cross-team friction. Go will often make decisions that incur individual overhead to reduce overall team overhead

                                  1. 5

                                    Not sure I agree on this.

                                    Compilation is not particularly fast, even compared to more more modern languages like Algol, its dependency management is a disaster, it’s error handling ignores the last few decades of lessons learned and the amount of code duplication it forces upon developers makes it hard to maintain.

                                    I think it does well in terms of helping Google’s requirements of having all code in a large mono-repo, and enabling people who have no practical experience to produce code.

                                    1. 2

                                      Whether or not they succeeded at being fast wasn’t my point (though my position is they did succeed). My point is the kinds of things they emphasized in language design. Russ Cox argues that compilation speed is one of the reasons they don’t have generics, for instance.

                                      Dependency management doesn’t matter with large teams in a mono repo, yeah, and the code duplication to me felt like it would be an enormous issue when I started but in practice, half a million lines of code later, it doesn’t come up nearly as much as you’d think.

                                      1. 2

                                        Compilation doesn’t have to be slow just because generics are involved, the author of D demonstrated that fairly well. I think this is rather an issue of generics not having been invented at Bell Labs (or decent error handling in this regard).

                                        I’m not sure why “dependency management doesn’t matter if you are a Google employee” should be a convincing argument for programming-in-the-large for the millions of non-Googlers out there.

                                    2. 2

                                      Structural typing specifically reduces cross-team friction.

                                      Can you talk about how structural typing accomplishes this?

                                      EDIT: Ah, I see you answered this in another thread.

                                  2. 3

                                    Languages are funny. I’d consider defer to be a bad idea elevated to a language feature, but it’s in the “good” group 😀

                                    1. 2

                                      Can you explain why you like this idea?

                                      1. 4

                                        Sure! Let’s say someone writes a library you use frequently but writes it such that it does not explicitly implement any interfaces, as the author of the above post prefers. Maybe you use a library called Beep like this:

                                        type Beeper1 struct { ... }
                                        func NewBeeper1() *Beeper1 { ... }
                                        func (b *Beeper1) Beep() { ... }
                                        

                                        You are writing your library, but want to support multiple implementations of Beepers. Maybe there’s another beeper (for a test, or another library, or something else) that also has a Beep() method. So you write your code to just expect

                                        type Beeper interface {
                                          Beep()
                                        }
                                        

                                        Now you can use the third party code, your code, your test code, etc, without having to change the third party code upstream to implement your interface.

                                        This is a super contrived example, but as your codebase and team grows larger, this becomes incredibly useful for reducing friction in having teams of engineers work together with minimal stepping on each other’s toes.

                                        Ultimately, I describe Go’s structural typing system to Python programmers like the static typing equivalent of Python’s “duck typing” principle, which is, if it looks like a duck and quacks like a duck, just treat it like a duck. Coming from statically typed languages that require you to list what interfaces a concrete instance implement, Go not requiring that dance felt like a huge reduction in friction to me.

                                        1. 2

                                          I guess, to me, it feels like a strictly worse approach than what Rust has with traits, or Haskell with typeclasses, because there’s no explicit guarantee that a “Beeper” is actually abiding by the contract of the “Beeper” interface. It could have a “Beep” method that actually nukes Vienna. There’s friction to implementing a trait or typeclass for a new type, but there’s also value in it. If I have explicitly implemented a trait, there’s documentation of the type’s usage in that context as well as of its conformance with the interface.

                                          1. 3

                                            A frequent pattern in Go to get some of that functionality if you want it is to write something like

                                            var _ InterfaceName = (*ConcreteType)(nil)
                                            

                                            which simply adds a compile time assertion that ConcreteType does indeed implement InterfaceName

                                            Certainly does nothing to constrain the behavior, but I’m super happy with that (optional) middle ground

                                            1. 3

                                              There exists a spectrum: let’s say that on one extreme, it’s maximum programmer friction with minimum risk of mis-use; and on the other extreme, minimum programmer friction with maximum risk of mis-use. Go puts a marker down closer to the latter extreme, judging friction to be a worse evil than risk for their context, and providing some affordances (like the one-liner jtolds mentions) to mitigate some of those risks via convention.

                                              I think no position on the spectrum is “strictly worse” than any other. It is a question of trade-offs to satisfy particular programming contexts or requirements. I think this is the same for any technical decisionmaking.

                                              Go makes a lot of decisions this way. (Not all, and there are warts for sure — but many.) I think it is a nice and refreshing change from where most languages (like Rust) decide to land, and I think Go’s success in the market proves it is a viable, or maybe even preferable, compromise-point for many users.

                                      1. 5

                                        A bit of shameless self-promotion, but I’m really proud of a library some friends at my company made called DBX: https://github.com/spacemonkeygo/dbx/

                                        It’s like SQLAlchemy, but completely type-safe and determined at compile-time. Many other Go ORMs aren’t quite as type-safe and do a lot of runtime reflection. DBX takes a schema upfront and generates multiple database backends and Go code to interact with your database in a type-safe way.

                                        It’s neat, check it out!

                                        1. 3

                                          Scott Hanselman just put together a great list (and did so for the previous two years also): https://www.hanselman.com/blog/The2017ChristmasListOfBestSTEMToysForKids.aspx

                                            1. 1

                                              @jtolds on previous submission: > Okay so for what it’s worth, logging in is actually required so we can measure how long it takes from the first time you read the problem until you solve it

                                              You need to say that somewhere in really big letters. I opened a problem because I was curious whether the problems were random little puzzles or closely tied to the sponsoring business. It looks like a five or ten minute puzzle, but I started making a cup of tea, planning a restaurant reservation, reading an interesting twitter thread, listening to a podcast…

                                              1. 1

                                                Since we’ve just enabled it such that anyone can view problems without logging in, we won’t be scoring by that criteria for any reason.

                                                You’re right, that should have been more prominent

                                              1. 2

                                                There’s no content here without signing up and giving an email address. It looks like it might be on topic, but it’s just lead gen.

                                                1. 1

                                                  Okay so for what it’s worth, logging in is actually required so we can measure how long it takes from the first time you read the problem until you solve it, but you’re right, this looks spammy. I’ll change it so it doesn’t require a log in.

                                                  1. 1

                                                    changed

                                                1. 4

                                                  To be honest, Go is a good language for just getting things done, but for something designed to be a server language, it has horrible runtime debuggability, introspection, and tracing features.

                                                  1. 4

                                                    I agree. The authors’ cultural biases and particular needs shaped very deeply the implementation. There are some people (myself included) who have worked on this somewhat, but it’s very hard to add it after the fact, so to speak.

                                                    1. 4

                                                      We wrote MonKit (https://github.com/spacemonkeygo/monkit) to add a lot of these features. Our Go programs are some of the best programs I’ve ever worked on for tracing, introspection, and debuggability. Certainly we’re still missing some things, but overall it’s a fantastic experience.

                                                      1. 4

                                                        It may not have Java’s remote debuggability but pprof exposed via an endpoint can often get you pretty close to what you need.

                                                        Although I’m probably a bit biased here since I almost never debug using an interactive debugger. I’m more of a printf and read the code kind of guy.

                                                        1. 3

                                                          I’m talking more about tapping into calls on a running system and running some stats and timing, maybe something like dtrace.

                                                          I can’t see a way to answer questions like “How many times is this function being called? how many errors am I getting here?” without recompiling.

                                                          I would love if Go provided a way for me to split the tracing and timing code out of my business logic.

                                                          1. 3

                                                            I’m not a Go fan, but if you’re on a platform with DTrace (e.g. SmartOS), I believe you can just use the pid provider to trace Go function calls. Because of the needlessly different calling convention you’ll probably need to learn about the stack structure to get arguments and return values, but it’s a start!

                                                            1. 11

                                                              Having written both the Solaris port and the SPARC port I use DTrace with Go all the time. It’s a massive PITA, but still better than not being to use DTrace.

                                                              Getting arguments is pretty simple.

                                                              #if defined(__sparcv9)
                                                              #define goarg0 *(long long*)copyin(uregs[R_O6]+0x7ff+176+0, 8)
                                                              #define goarg1 *(long long*)copyin(uregs[R_O6]+0x7ff+176+8, 8)
                                                              #define goarg2 *(long long*)copyin(uregs[R_O6]+0x7ff+176+16, 8)
                                                              #elif defined(__amd64)
                                                              #define goarg0 *(long long*)copyin(uregs[R_RSP]+8+0, 8)
                                                              #define goarg1 *(long long*)copyin(uregs[R_RSP]+8+8, 8)
                                                              #define goarg2 *(long long*)copyin(uregs[R_RSP]+8+16, 8)
                                                              #endif
                                                              

                                                              There are other problems, however.

                                                              The main problem is the lack of debugging information. Because of the way Go binaries are built it’s impossible to add CTF to Go binaries the regular way you’d do in C. So I added CTF support directly in the Go linker. However, I quickly learned that virtually all Go binaries exceed the CTF limits for a single object file (number of symbols and so on). You’d have to split, at least in conceptually, in memory, Go object files into multiple object files so they’d fit into CTF limits.

                                                              This is very, very difficult to do under the current toolchain implementation. Truth be told, if CTF were more widely used I’d do it, but I bet the number of people who run DTrace or mdb(1) on Go binaries can be counted on fingers, and I am probably the largest user, and I have my own workarounds.

                                                              There is the second aspect of this story, and that is that on Oracle Solaris, DTrace and mdb(1) got native DWARF support (which the Go linker has generated for a very long time). It turns out that if you restrict the DWARF in the binaries to only the subset that can be expressed in CTF, it’s not actually any larger than CTF. I suggest illumos and FreeBSD move to the same technique, as DWARF is ubiquitous.

                                                              Apart from the lack of debugging information, the biggest problem is that in DTrace there is no language-level concept of goroutines. For example, even if you can easily extract a pointer to some goroutine, you can’t get a stack trace for that goroutine easily. This is solvable with DTrace providers, but I have not yet written one.

                                                              There are also more minor nuissances like the fact that the set of expressable identifiers in DTrace and mdb(1) is smaller than the set of identifiers expressable in Go. In particular, you can’t refer directly to most Go identifiers in DTrace or mdb(1). Usually when I need this I patch the linker to implement some mangling scheme or use the symbol address.

                                                              The good news is that I do have plans to improve support for DTrace in Go, like writing a provider and so on. The bad news is that there is no ETA.

                                                              Oh, and by the way:

                                                              needlessly different calling convention

                                                              The calling convention is not needlessly different. It has been different for very specific technical reasons and it’s obvious from our previous discussions that you, and in general Joyent, has an anti-Go-implementation bias here.

                                                              There are some not-very-concrete plans put forwards by some people to change it to make it more compatible with the System V ABI (it can never be fully compatible), but it’s actually rather difficult to do and the priorities have been elsewhere. Of all the people who work on Go I am probably the one who recognizes the need for this the most, so there is some rather good chance that I might tackle this some day, but as always, there have been other priorities.

                                                              1. 2

                                                                it’s obvious from our previous discussions that you, and in general Joyent, has an anti-Go-implementation bias here

                                                                Calling my technical opinion a “bias” feels like a rhetorical technique to suggest that I couldn’t possibly have a legitimate criticism. Additionally, though I work at Joyent (even with Keith, now retired), I don’t think it’s fair to characterise my opinion of Go as the position of an entire company.

                                                                We are a broad church and have quite a few engineering staff presently experimenting with Go, some of whom are attempting to actively engage with the Go community on illumos-specific matters. At least one of us is working on a new mdb module for Go, and you can see a bunch of recent activity in our Triton/Manta SDK for Go. We’re even working on an evolving set of engineering practices specifically for writing software in Go.

                                                                I bet the number of people who run DTrace or mdb(1) on Go binaries can be counted on fingers, and I am probably the largest user

                                                                That may have been true in the past, but I don’t imagine it will be true in the future!

                                                      1. 3

                                                        tl;dr: in the long run, everyone has their moments of wealth and poverty, but at any given time, the distribution of wealth is exponential.

                                                        1. 8

                                                          Within that experiment, sure. Unfortunately in the real world, having more money allows you to acquire more money much easier (e.g., not having to pay the bank interest for loans, actually having money you can invest, etc.), so there’s a bit of a runaway problem where fairness won’t necessarily come in the end without intervention.

                                                          1. 1

                                                            Practical forms of “intervention” include exponential dilution via children, marriages, etc.

                                                            You can get around this with e.g. primogeniture but this is evolutionarily sub-optimal so most people don’t. Even the oldest banking families around today aren’t really all that old on a historical scale.

                                                            1. 2

                                                              I can see this could prevent long-lasting dynasties but when few hold money it’s still bad for the economy even if it was just for a single generation. The poor spend a much larger portion of their income than wealthy even in contrived environments. I do think the property rights (yes unlike the founding fathers I do view property as a right) of the few do not outweigh the rights of life liberty and the pursuit of happiness. If the income inequality is so extreme that those basic rights are infringed I think it is a responsibility of the government to normalize that effect.

                                                        1. 8

                                                          This reminds me of the problem Mitzenmacher’s “power of two choices” load balancing scheme solves [1]. The problem is that basically, if you throw n balls into n bins completely at random, some bins will have no balls while some bins will have more than one. Ultimately, the difference between the bin with the least amount of balls and the bin with the most goes up with log(n)/log(log(n)). This can actually become a pretty significant effect with large n.

                                                          So, yeah, if you simply distribute money randomly, the difference between the people with the most money and the people with the least money won’t be very well load balanced. What’s more, in the case of this thought experiment, there’s an additional effect which is that some rounds, some players are out of money and can’t redistribute, which reduces the overall flow of money that round.

                                                          Mitzenmacher’s power of two choices load balancing scheme is to randomly choose two bins, then place the next ball into the bin with the lesser amount of existing balls. With this strategy, the maximum skew is more like log(log(n))/log(2), which is way better distributed.

                                                          I have no idea on how that load balancing observation could be used to create better wealth inequality policy.

                                                          [1] https://www.eecs.harvard.edu/~michaelm/postscripts/mythesis.pdf

                                                          1. 1

                                                            I have no idea on how that load balancing observation could be used to create better wealth inequality policy.

                                                            Well, it would seem that a random distribution creates clumps and hidden money (all money past 1$ is “hidden”).

                                                            So, a system where everyone gets the money would be a better way. Like UBI. The payment of goods and services may still be skewed, but UBI would not lock people out of it. The other half would be a progressive tax to lower overall effectiveness of higher end and rebalance the monetary system.

                                                          1. 7

                                                            Ugh, I added a new disclaimer to the top of the article I’m linked to from this list (update 2 on http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/)