1. 7

  2. 4

    To be honest, Go is a good language for just getting things done, but for something designed to be a server language, it has horrible runtime debuggability, introspection, and tracing features.

    1. 4

      I agree. The authors’ cultural biases and particular needs shaped very deeply the implementation. There are some people (myself included) who have worked on this somewhat, but it’s very hard to add it after the fact, so to speak.

      1. 4

        We wrote MonKit (https://github.com/spacemonkeygo/monkit) to add a lot of these features. Our Go programs are some of the best programs I’ve ever worked on for tracing, introspection, and debuggability. Certainly we’re still missing some things, but overall it’s a fantastic experience.

        1. 4

          It may not have Java’s remote debuggability but pprof exposed via an endpoint can often get you pretty close to what you need.

          Although I’m probably a bit biased here since I almost never debug using an interactive debugger. I’m more of a printf and read the code kind of guy.

          1. 3

            I’m talking more about tapping into calls on a running system and running some stats and timing, maybe something like dtrace.

            I can’t see a way to answer questions like “How many times is this function being called? how many errors am I getting here?” without recompiling.

            I would love if Go provided a way for me to split the tracing and timing code out of my business logic.

            1. 3

              I’m not a Go fan, but if you’re on a platform with DTrace (e.g. SmartOS), I believe you can just use the pid provider to trace Go function calls. Because of the needlessly different calling convention you’ll probably need to learn about the stack structure to get arguments and return values, but it’s a start!

              1. 11

                Having written both the Solaris port and the SPARC port I use DTrace with Go all the time. It’s a massive PITA, but still better than not being to use DTrace.

                Getting arguments is pretty simple.

                #if defined(__sparcv9)
                #define goarg0 *(long long*)copyin(uregs[R_O6]+0x7ff+176+0, 8)
                #define goarg1 *(long long*)copyin(uregs[R_O6]+0x7ff+176+8, 8)
                #define goarg2 *(long long*)copyin(uregs[R_O6]+0x7ff+176+16, 8)
                #elif defined(__amd64)
                #define goarg0 *(long long*)copyin(uregs[R_RSP]+8+0, 8)
                #define goarg1 *(long long*)copyin(uregs[R_RSP]+8+8, 8)
                #define goarg2 *(long long*)copyin(uregs[R_RSP]+8+16, 8)

                There are other problems, however.

                The main problem is the lack of debugging information. Because of the way Go binaries are built it’s impossible to add CTF to Go binaries the regular way you’d do in C. So I added CTF support directly in the Go linker. However, I quickly learned that virtually all Go binaries exceed the CTF limits for a single object file (number of symbols and so on). You’d have to split, at least in conceptually, in memory, Go object files into multiple object files so they’d fit into CTF limits.

                This is very, very difficult to do under the current toolchain implementation. Truth be told, if CTF were more widely used I’d do it, but I bet the number of people who run DTrace or mdb(1) on Go binaries can be counted on fingers, and I am probably the largest user, and I have my own workarounds.

                There is the second aspect of this story, and that is that on Oracle Solaris, DTrace and mdb(1) got native DWARF support (which the Go linker has generated for a very long time). It turns out that if you restrict the DWARF in the binaries to only the subset that can be expressed in CTF, it’s not actually any larger than CTF. I suggest illumos and FreeBSD move to the same technique, as DWARF is ubiquitous.

                Apart from the lack of debugging information, the biggest problem is that in DTrace there is no language-level concept of goroutines. For example, even if you can easily extract a pointer to some goroutine, you can’t get a stack trace for that goroutine easily. This is solvable with DTrace providers, but I have not yet written one.

                There are also more minor nuissances like the fact that the set of expressable identifiers in DTrace and mdb(1) is smaller than the set of identifiers expressable in Go. In particular, you can’t refer directly to most Go identifiers in DTrace or mdb(1). Usually when I need this I patch the linker to implement some mangling scheme or use the symbol address.

                The good news is that I do have plans to improve support for DTrace in Go, like writing a provider and so on. The bad news is that there is no ETA.

                Oh, and by the way:

                needlessly different calling convention

                The calling convention is not needlessly different. It has been different for very specific technical reasons and it’s obvious from our previous discussions that you, and in general Joyent, has an anti-Go-implementation bias here.

                There are some not-very-concrete plans put forwards by some people to change it to make it more compatible with the System V ABI (it can never be fully compatible), but it’s actually rather difficult to do and the priorities have been elsewhere. Of all the people who work on Go I am probably the one who recognizes the need for this the most, so there is some rather good chance that I might tackle this some day, but as always, there have been other priorities.

                1. 2

                  it’s obvious from our previous discussions that you, and in general Joyent, has an anti-Go-implementation bias here

                  Calling my technical opinion a “bias” feels like a rhetorical technique to suggest that I couldn’t possibly have a legitimate criticism. Additionally, though I work at Joyent (even with Keith, now retired), I don’t think it’s fair to characterise my opinion of Go as the position of an entire company.

                  We are a broad church and have quite a few engineering staff presently experimenting with Go, some of whom are attempting to actively engage with the Go community on illumos-specific matters. At least one of us is working on a new mdb module for Go, and you can see a bunch of recent activity in our Triton/Manta SDK for Go. We’re even working on an evolving set of engineering practices specifically for writing software in Go.

                  I bet the number of people who run DTrace or mdb(1) on Go binaries can be counted on fingers, and I am probably the largest user

                  That may have been true in the past, but I don’t imagine it will be true in the future!