1. 4

    From the ‘About’ page …

    This is a fork of gddo, the software which previously powered the now-defunct godoc.org, operated by Drew DeVault. The source code is available on sr.ht.

    Anyone here have context around the situation that led to the fork? I must have entirely missed this (but that’s not surprising as I only tinker with Golang).

    1. 20

      Godoc.org used to be an independent, open source site that Google bought just to cancel and replace with proprietary bloat that takes >10x longer to render, and won’t display documentation for except for licenses deemed acceptable by Google, in particular it rejects public domain and WTFPL. Some people dislike that and decided to do something about it.

      I never cared about “silly” licenses like the WTFPL, but now I am tempted to license all my code under WTFPL. Google won’t allow the use of WTFPL code internally, which for me is absolutely great.

      1. 31

        Godoc.org used to be an independent, open source site that Google bought just to cancel and replace with proprietary bloat …

        You posted the same lies on HN. Please stop.

        Longer reply there: https://news.ycombinator.com/item?id=29314711.

        1. 6

          But the part about licenses is correct.

          1. 3

            Something like this would be better then, if one wanted to be inflammatory but more correct:

            Godoc.org used to be an independent, open source site that Google agreed to take stewardship over, just to cancel and replace with proprietary bloat that takes >10x longer to render, and won’t display documentation for except for licenses deemed acceptable by Google, in particular it rejects public domain and WTFPL. Some people dislike that and decided to do something about it.

            EDIT: I see from the rest of the hacker news discussion that the replacement isn’t proprietary. I can’t comment on the bloat/performance aspect, new site seems fast to me. That really only leaves the licensing stuff.

            1. 6

              If people haven’t granted the right to display the documentation then sites probably shouldn’t display it, right?

              1. 1

                Still not really true. First off, a public domain dedication is not a licence. Second, CC0-1.0 is accepted.

            2. 6

              WTFPL is problematic in general because the public domain doesn’t exist in all jurisdictions, which is why CC-Zero exists. If your goal is to build free software that other people can reuse, I wouldn’t recommend it.

              If your goal is to spite Google, use something like AGPL instead.

              1. 1

                WTFPL isn’t a public domain dedication. It’s an informal statement.

                1. 1

                  the public domain doesn’t exist in all jurisdictions

                  Which is exactly the reason cited for the creation of WTFPL. It’s basically CC-less-than-zero (and it predates CC0 by several years).

                2. 5

                  … and I’ll bet doesn’t work well on non-Chrome (maybe non-Firefox?) browsers either - and thus not on Plan 9. Whereas godocs.io seems to work nicely on Netsurf.

                  Edit: To my surprise, it (somewhat) does, albeit with horrendously broken layout.

                  1. 7

                    Can confirm (from my experience) on Firefox that pkg.go.dev has some… paper cuts, whereas on Chrome I haven’t run into any issues. godocs.io on the other hand has worked flawlessly on both and is my go to.

              1. 5

                Going to give learning Clojure a shot. I’ve been diving into Emacs some more so kind of excited to learn a more widely used lisp.

                1. 6

                  Good luck! I’ve used Clojure as my main language at work for nearly 2 years now, and still liking it. I also use Emacs—I would recommend configuring flycheck-clj-kondo, and lsp-mode (clojure-lsp) for jumping around. FWIW my rather simple Clojure-related Emacs config is here.

                  1. 1

                    This is fantastic! Thanks @stig

                  2. 1

                    Coming from which language, out of curiosity?

                    1. 2

                      I write Java for the day-job, and on the side I’ve been mostly doing Rust and Go for a handful of years.

                  1. 1

                    Nice writeup :)

                    1. 2

                      More generally, I find that ctags are surprisingly underpowered even for robust semanticless searcher. I believe it should be easy to build a significantly more powerful tool (comment with details).

                      1. 2

                        +1 in fact @matklad it was your comment that has me thinking about this a bit more. I’ve been messing around with Tree Sitter and was doing some initial comparisons with ctags which is how I came across this interesting behaviour. Hopefully more to come on this subject :)

                        1. 2

                          This is exciting!

                      1. 5

                        Heh, I personally just implement the thing myself (1, 2) :)

                        More practically, I think there are these choices available:

                        • use smart IDE, like IntelliJ
                        • use editor with LSP support (and bug LSP, editor and server developers to improve support for this, as they don’t allow you to filter your code vs library code, or test vs main code)
                        • produce symbol index as a part of build process (Google’s kythe).
                        • use offline regex based index (ctags). This works well enough, but might not be super precise (as regexes can’t parse programming languages), and have a drawback that index rebuild must be triggered manually.

                        The next two options are a straightforward (Christmas holidays scale) extensions to the last one, but I don’t think they are available as a popular stand-alone tool.

                        • online indexing – listen for the file changes and apply an incremental diff to the index, rather than recomputing it from scratch
                        • plug tree sitter instead of regular expressions for richer and more correct indexes.
                        1. 2

                          Sourcegraph has an open source core, for a standalone tool: https://github.com/sourcegraph/sourcegraph

                          1. 2

                            But does the core actually has indexing capabilities? I don’t know, but I think they are consumers of LSIF (a feature of LSP) and ctags, rather then producers?

                            GitHub’s semantic might be closer to what I am talking about (but, again, haven’t looked close at it):

                            And, naturally, IntelliJ is open source, that would be (well, it is :) ) the first place for me to look how to do language-aware tooling:

                            1. 2

                              We’ve run the OSS sourcegraph indexers internally and they worked quite well. It isn’t currently running (just for lack of use - mostly). But unless they changed the OSS offering recently, it definitely works as an indexer.

                              1. 2

                                I was thinking about this comment more after replying and I think my first reply is confusing/wrong. I was thinking, as an end-user, does open source Sourcegraph have code search capabilities.

                                You’re right - the core consumes LSIF.

                                https://docs.sourcegraph.com/code_intelligence/references/indexers https://lsif.dev

                            2. 1

                              I really wish kythe has better open source support. The docs are out of date and its hard to get started without a viable frontend.

                              1. 1

                                plug tree sitter instead of regular expressions for richer and more correct indexes.

                                I assume you’re referring to this tree sitter? Looks like an awesome project!

                                @matklad are you aware of any projects using tree sitter for symbol tagging/indexing? And, with your work on rust-analyzer (and thank you for all you’ve put into that) do you have any thoughts on where effort is best placed to improve code indexing outside of big IDEs like IntelliJ? I.e: working more on LSPs to expand support for indexing and searching, and ensure they are robust and work well on large codebases vs building a modern ctags-like tool that is perhaps based on tree sitter.

                                1. 2

                                  @matklad are you aware of any projects using tree sitter for symbol tagging/indexing?

                                  I think GitHub’s semantic does that (although not sure), but it tries to go way beyond simple indexing. In terms of making a dent, I think if you want to support a specific language, you really should push on the LSP implementation for that language making server more robust & fast, making client more powerful, and making the protocol more capable and less bet.

                                  If you want to improve the base line support for all languages, I think a tree-sitter based map-reduced indexer could make a meaningful difference. Basically:

                                  • some infra to write mappers which take tree-sitter’s syntax tree and output the list of (symbol name, symbol kind, symbol span) triples (with maybe some extra attributes)
                                  • some fuzzy-search index for symbol names (if using rust, just take the fst crate (or copy-paste symbol_index.rs from rust-analyzer))
                                  • a driver which can:
                                    • process a bunch of files offline embarrassingly parallel
                                    • incrementally update the index when files change (that is, if file x changes, remove all x keys from the index, and then re-add the keys after the mapper step).
                                  • some client API – LSP protocol for easy integration, custom search protocl for more features (streaming & filtering) or just a CLI for unix way
                                  • an optional persistence layer, to load the index from disk (but you most definitely can live with purely in-memory index for a long time, don’t implement this from the start, postpone)
                                  • some pluggability, so that it’s not you, but your users who maintain grammars & mappers.
                                  • a kind of a feature creep, but why not – a tri-gram index on top of driver/mapper infrastructure, to speed-up text-based greps.
                              1. 8

                                How do you index code in your projects?

                                I used to use tags and now I use an LSP client (the one shipped with Neovim v0.5). I find that trying to use the CLI to discover semantic information about your code doesn’t work past a certain amount of lines of code. I still regularily make use of grep, both in a terminal and with a “fuzzy” matcher (denite) but the larger the project the less I’ll use it. For working on Firefox and Linux I prefer using https://searchfox.org/ and https://elixir.bootlin.com/linux/latest/source though.

                                Additionally, from lurking round the web, I get the impression that ctags (and alternatives) aren’t really used anymore or that there’s now a better way of supporting jump-to-symbol in Vim. Is this accurate and, if so, what’s the new alternative?

                                I would say that it is accurate and that the alternative is LSP clients and servers.

                                1. 3

                                  I would say that it is accurate and that the alternative is LSP clients and servers.

                                  Personally, I find it annoying that there was such a quick switch to something that is still as unstable and raw as LSP. Setting up a server is often not easier then using tags, crashing is not infrequent and they are usually just tested with VSCode, which allegedly isn’t a faithful implementation of the specification. Nevertheless, so many languages and work environments have deprecated whatever was working before to support LSP, which at least for me has broken more things that helped me.

                                  It’s a classical example of where more though should have been put into it, but the first working implementation came quicker.

                                  1. 4

                                    Setting up a server is often not easier then using tags, crashing is not infrequent

                                    The bad news is that it’s not just immaturity. A language server is necessary a couple of orders of magnitude more complex than ctags-like approaches (for one, it needs to integrate with project’s build system). Even if perfectly implemented, it would be less reliable and less robust than simpler approaches.

                                    That being sad, as a server-implementer I agree that the protocol could have been designed better.

                                    1. 1

                                      kinda silly but I wonder if you could build like… a ctags LSP shim or something for people who want to manage ctags but are using an LSP-based system

                                      1. 1

                                        totally! This is one of the things I’ve suggested in my other comment: https://lobste.rs/s/ujr9mg/how_do_you_index_code_your_projects#c_buj3rg

                                      2. 1

                                        This has certainly been inline with my experiences with LSPs (mostly for Rust, Go, and Python). Great concept, and amazing when they work but very finicky with frequent crashes.

                                      3. 3

                                        It’s a classical example of where more though should have been put into it, but the first working implementation came quicker.

                                        This is how I feel about pretty much all the code indexing stuff I’ve used (GNU Global, Ctags and its seemingly unending variants, Cscope). They’re all janky and have been janky for ages.

                                        1. 1

                                          The only tooling I’ve had work reliability in terms of indexing/intelligence has been if the editor/IDE does it itself. Visual Studio, IDEA, and good Emacs major modes are the platonic ideal here.

                                        2. 2

                                          Nevertheless, so many languages and work environments have deprecated whatever was working before to support LSP

                                          Interesting, what are these environments and what did they remove support for? My knowledge is limited to (Neo)Vim and tags are still available and supported there.

                                          they are usually just tested with VSCode

                                          Yes, that is true. There’s even a great rant by an LSP client maintainer where he points out several issues with LSP, among which the fact that most LSP servers are bad citizens.

                                          It’s a classical example of where more though should have been put into it, but the first working implementation came quicker.

                                          Wasn’t the first working implentation ctags (or whatever was there before)? It’s not like jump-to-definition is something completely new, there had already been several approaches by the time LSP was created. The fact that it became so popular so fast IMO proves that it has multiple advantages over the SOTA, despite the bad things in its design (UTF-16, really???).

                                          I think what’s really required now is to create a conformance testsuite, fuzzers and other tools to improve the quality of current implementations.

                                          1. 2

                                            Interesting, what are these environments and what did they remove support for?

                                            I was thinking about if the other way around: Go and Haskell tools for development are not being worked on any more (and at least in the case of Go is now non-functional), and instead everyone is focusing on LSP, which is still not stable.

                                      1. 0

                                        nice write up :)

                                        1. 6

                                          https://joshleeb.com - some old blog posts and a really outdated ‘about me’ section

                                          1. 3

                                            Tutoring high school kids the basics of python and web programming at NCSS

                                            1. 3

                                              Tidying up a crate (mtls-messages) that’s part of micro-tls: a no_std rust impl of tls for embedded devices. Made some good progress getting the encoding and decoding right with no copies but still got a long way to go before tls is properly implemented

                                              https://github.com/joshleeb/micro-tls

                                              1. 13

                                                Reading this made realize I don’t see myself in any of the 3 groups. I’m probably closest to a “maker”, however, the author seems to tie that somewhay to UI and “front-end”.

                                                I work on line-of-business systems that tend to be large and complex. In that, I most consider myself a modeler. I’m primarily about modeling systems, real and imagined, into a working program, but not through the lens of a UI (or a relational database model).

                                                Instead, I see functional requirements for a large, complex system being implemented in an object-oriented “domain model”.

                                                Am I just a dinosaur, and lonely group of one, these days?

                                                1. 40

                                                  Am I just a dinosaur, and lonely group of one, these days?

                                                  I think the author is just wrong. People really like putting things into neat little boxes, but I would guess there are more than 3 types of programmers out there.

                                                  1. 15

                                                    I’m going to go one further and say these 3 “tribes” are just different ways of thinking that anyone can adopt at any time, or depending on the situation. I could plausibly claim membership of a different tribe every hour of the working day.

                                                    1. 7

                                                      To me, that flexibility is one of the most rewarding things of the profession: I can inhabit the mindset that best fits the problem I’m currently faced with, and I find my best ideas often by flipping to another perspective.

                                                      1. 5

                                                        Indeed, in fact the thing that would most likely make me say someone isn’t a “real programmer” is not their membership in the “wrong tribe”, but rather their inability to switch between all three of these modes as appropriate.

                                                        1. 2

                                                          I think the difficulty lies in that the initial discovery of each tribe is met with the revelation of, “ah, I’ve found the way to build software.” Then you sit with it for a few years and start to see the weak spots.

                                                      2. 13

                                                        Agreed. This is an arbitrary distinction and many programmers don’t tend to fall neatly into one category or the other. As a Scheme implementor, I believe “camps” 1 and 2 are not mutually exclusive, and in fact there’s a long history of programming language implementation research that tries to bridge the gap between beautiful mathematical constructs and how to make a machine execute it as fast as possible. In fact, this is the most interesting part of PL implementation!

                                                        1. 3

                                                          not to mention programming language and library design as the user interface you present to other developers.

                                                          1. 4

                                                            Absolutely, that’s more the “craft” or engineering aspect of programming, which fits better in the third “tribe” from the article. You need to be able to work on and balance all three aspects, really :)

                                                        2. 4

                                                          I agree with that. Furthermore, categorizing in general throws away information and that may or may not be suitable depending on the context. Perhaps a better approach would be to identify different traits of programmers and treat each of them as a continuous variable. Based on that it would be possible to categorize less arbitrarily (e.g. by finding clusters in the data). This would be a more data-driven approach and one would obviously need a representative dataset for that.

                                                        3. 11

                                                          No. As an experienced software person, you simply have to divide your time into multiple heads-paces.

                                                          Sometimes you can really appreciate something like this:

                                                          qs([H|T])->qs([X||X<-T,X<H])++[H]++qs([X||X<-T,X>=H]);qs([])->[].
                                                          

                                                          Other times, you might just do lists:sort(A++B), and yet still other times you would be like no you really can get away with lists:merge(A,lists:sort(B)).

                                                          That’s ok. It doesn’t make you alone, but it can be lonely: Most teams in my experience have just one of you, and even in a large company there are likely to be very few of you.

                                                          The article may be written by someone just discovering these different head-spaces exist and jumping to the conclusion that they fit a taxonomy, instead of more like a venn-diagram of considerations that go into implementation and design. That’s unfortunate, but it doesn’t mean there isn’t some good stuff to think about.

                                                          1. 5

                                                            I think this post would make more sense if it focused less on UI and more on shipping things. This would also avoid the awkwardness of writing “Bret Victor” below this:

                                                            How the code interacts with humans is a separate consideration from its implementation. Beautiful code is more important than beautiful UI.

                                                            I think I am personally a bit of all three, naturally trending more towards 1 but trying to be more like 3.

                                                            1. 3

                                                              Yeah, I had a similar feeling when reading this. I have mostly worked on HPC systems which are used for scientific simulations, and my primary concern is helping scientists do that. I write software in the course of building and maintaining those systems, but if I could make “simulate scientific problems” work without writing any software, I’d do that instead.

                                                              Or, as a friend of mine put it: computers are just a means, not an end.

                                                              1. 1

                                                                Reading this made realize I don’t see myself in any of the 3 groups.

                                                                I’m glad I came to the comments and saw this as the first line of the first comment. I absolutely feel the same way.

                                                              1. 3

                                                                Finishing off an initial build of a note taking curator/renderer (needs way more polish) and having a 2nd attempt at a tls1.3 client written in rust for no_std (embedded) targets

                                                                1. 1

                                                                  Your “design doc” readme is quite impressive, detailed yet still readable and interesting (at least to me). Is it customary for you to start projects with such a writeup & analysis? And/or do you keep coming back to it regularly to keep it up-to-date throughout the whole development time of the project, or is it more of a one-off, or a loose scratchpad for ideas?

                                                                  1. 2

                                                                    Thanks! I’m planning on turning it into a blog post in the coming week.

                                                                    I usually write one up at the start of projects that I expect to be more than just hacking on something over a day or two. I find it helps to keep me on track with what I set out to do, and reduces the chance I’ll jump to another new project. I try to update it as I go but it’s usually not the first thing on the list so these docs can sometimes get quite out of date but at that point I still find them useful to jog my memory of what I was thinking.

                                                                1. 2

                                                                  This is amazing. And a great write-up too!