1. 30

  2. 32

    For historical context, I completely agree. For more modern things, the academic paper format has become so stylised and rigid that I’d recommend reading the blogs of people who write papers, rather than the papers, if you want to understand what they’re doing.

    1. 4

      Papers are more discoverable, though, specially since they all reference related papers. And they provide starting points, where blogs usually don’t.

      Coming to someone’s blog is often like “The latest update on Cool Thing since [last post] is we were able to reprong the cohex-trie into being n-profligate by …” and then you have to do a depth-first traversal of the links to figure out what any of those words mean or what is going on.

      1. 2

        I’d also throw the Amazon Builders’ Library into the mix. I’ve found the write-ups they put up to be quite interesting and I’m glad they’re sharing these. There doesn’t seem to be an RSS feed one can subscribe to, but there are ways to stay up-to-date on new releases.

        1. 2

          I’ve done this but it always feels like… Cheating? :) I dunno.

          1. 21

            The papers I write always suffer from at least one out of:

            • The venue only accepts things written in a particular shape (they never document this, but the review process is driven by implicit bias and everyone knows what a paper for X should look like) and so I have to shape things to meet expectations, rather than to communicate effectively.
            • The page limit prevents me from properly explaining the details.
            • The rigid deadline means I have to rush some experiments and not give proper root-cause analysis or miss some possible approaches that I don’t discover until later.

            I write papers for two reasons:

            • To get points in stupid bibliometric games that people care about for research jobs.
            • To go to conferences and meet people I might want to hire.

            I write publish blog posts, tech reports, and open source repositories to communicate the results of my research.

            I say this as someone with a bunch of publications in top-tier venues, including some that won awards.

            1. 5

              This feeds my sneaking suspicion that “You should write papers” is more a mental toughness test and, dare I say it? Virtue signaling than an actual thought that this is a good way to learn new information about computer science :)

              1. 7

                Academic papers have a different audience than blog posts. They’re written for other experts in the field (unless it’s a review article, which are more for introducing new researchers to the relevant literature). They are rarely expository about concepts that were developed elsewhere in the literature.

                Different audience, different content.

                1. 2

                  I think that’s quite an optimistic framing.

                  I did my PhD at Swansea, where people rarely sent papers to top tier venues. I then went to Cambridge, where the attitude was to always send things to top-tier venues or not bother. Once you start aiming for the top venues, you learn that they have a particular shape of paper that they accept and that fitting these expectations influences acceptance a lot more than the quality of the work. It’s not about writing for experts in the field (the reviewers often have a fairly broad background within a sub field) it’s about writing the shape of paper that they expect as gatekeepers. Part of that often involves making things over complicated so that reviewers don’t say that the contribution was too small.

                  In contrast, when people write blogs they want to engage the maximum number of people.

            2. 3

              I read papers a bit (related to my work) and I highly recommend seeking out the conference video, looking for blog posts, etc. along with looking through the paper. These are a great way to give you a leg-up in getting into a paper, and not cheating at all in my opinion! Having the author pick out the important bits and frame things in their own way can be really helpful, especially if you’re not used to papers in a given subfield.

              Also in general, don’t feel the need to read from top to bottom (edit: see hwayne’s comment). And if there are mathematical justifications you can often skip those if you are more interested in the more practical outcomes.

          2. 10

            I feel like this is one of those personal discipline issues: Whenever I try to read most CS papers, I find them utterly impenetrable.

            I can usually get through the summary fine, but the actual body of the paper might as well be written in greek.

            I’ve even tried the tack of saying “OK, I will stop and go look up each new term as I see it” but then I end up losing the thread and coming back unable to absorb the actual containing thought.

            I KNOW this is about me and not about the papers, and I suspect there are lots more like me who understandably aren’t willing to be so open about their foibles :)

            1. 6

              In my experience it depends on the topic. In general, the more theoretical or mathematical, the more opaque. I can’t follow most papers on PLT, both because they use ultra-concise notation and I quickly lose track of what ξ represents, and because they’re assuming understanding of books’-worth of concepts like lambda calculus.

              On the other hand, I’ve been reading a lot about garbage collectors and memory allocators, and those papers are pretty clear. It may take some careful re-reading to understand a clever algorithm, but it’s worth it. Ditto for papers on data structures like hash tables or tries.

              1. 2

                What always gets me about the math heavy papers is how often they double down to explain the easy stuff (e.g. matrix multiplication) in great detail and then skim over the hard stuff (e.g. how to calculate the matrix). Half the CS paper’s I’ve seen have some variation of the equation

                Θ = Σaᵢbᵢ

                The values of vectors a and b will then be buried in prose, often with one vector only being referred to by a proper noun that does not occur in any of the refernces. What is the presumed audience that can instantly figure out the complicated incantations the authors vaguely allude to, but need a daily reminder on how to calculate a dot product?

                1. 2

                  The thing that bugs me the most about the math notation is having to represent everything by a single letter (Roman or Greek.) I quickly lose track of what B and ε refer to. Meanwhile, every language since early BASIC has allowed multi-letter variable names.

              2. 4

                It’s not you. It’s inherent to the process. Academic papers are written by domain experts for other domain-adjacent experts. Much of the content is often focused on very nuanced details that only confuse a relative outsider. It can take years of full time reading and experience in the domain to be able to read and understand the average (or at least the well written) academic papers from that domain.

                Unfortunately, there’s very little incentive for academics to write slightly more accessible expositions. There’s incentive to write layman summaries and have their University’s PR group publish them, but very little incentive to write anything between those two extremes. Unless you count writing textbooks, but those usually aren’t free (as in beer).

                1. 3

                  I disagree. I’ve read some incredible papers that were rejected from top-tier venues because the authors explained their ideas so clearly that they seemed obvious.

                  A lot of the time, problems are only hard because they are poorly understood. Once you properly mail down the problem that you’re trying to solve, you constrain the set of solutions such that it becomes obvious. My favourite example of this is the spin lock paper, which started with a simple test and set lock and went through the cache coherency overhead and proposed increasingly good refinements. The thing I loved about this paper was that I came up with each of their approaches before they explained it because their analysis of the flaws in each previous approach made it obvious. It would be very hard to publish something like that now because reviewers would say it was too obvious.

                2. 3

                  They’re also usually horribly written. As someone elsewhere in the thread pointed out, papers are written to score points in a game, not to convey information. (Unscalable 2-column TeX PDF should have been our first clue…) For many articles, my reward for figuring out how to evade the paywall and plowing through the jargon and greek letters is to find obvious bugs in the code/pseudo-code.

                  All of which is to say: You’re not alone, far from it. And I don’t think you’re missing much. These days I skip most papers and just read blog articles.

                  1. 6

                    arXiv papers can be read as HTML if you replace the X with a 5: https://ar5iv.labs.arxiv.org/html/2010.00029

                3. 8

                  A lot on why you should read CS papers and which papers you should read, but not how to read them. First rule: read the abstract, skim the introduction, read the conclusion, then decide if it’s worth reading the rest of the paper.

                  1. 2

                    Those are good steps. I also always read “related work” while skimming the implementation and limitations.

                    Related work of a few papers gave me a start on a whole survey of that sub-field. How they do the implementation hints at if what they did even matters in the real world. Limitations will do that while telling you the next, research problem to work on. Also, what tech to mix and match it with since its limitations will be the advertised strengths of other works.

                  2. 4

                    There are basically two kinds of computer science papers:

                    1. “I did something so monumental, so high-impact, that it absolutely needs a formal paper.” You should read these ones.

                    2. “I did something very novel, which builds on other computer science papers in interesting ways, and presents some interesting challenges for the future, but hasn’t had any impact yet.” These are probably not worth the time to read, since most of them have ideas that have not had any impact yet. If you ever do a deep dive into the topic for a specific reason, then it makes sense to read.

                    Like startups, or corporate projects, or anything else in existence really, there are specific criteria that allow CS papers get published that are probably not all that well aligned with “how good is the idea.” This can make coming in as an outsider feel a bit like crashing a party of strangers.

                    1. 3

                      Based on this thread, what we really should have are working-programmer-friendly summaries / explanations of academic CS papers.

                      1. 8

                        The Morning Paper blog is this!

                        Does anyone else know of similar resources?

                        1. 4

                          It Will Never Work in Theory introduces more than summarizes but does turn up great stuff, and The Shape of Code takes quantitative studies as springboards to go some interesting places.

                        2. 5

                          The article suggests “Papers We Love”, which I’ve also enjoyed in the past. It’s usually much better written than the original papers it covers.

                          1. 2

                            Isn’t Papers We Love video-only? I could be wrong. But, I still want something to read. It’s still a great resource though.

                            1. 1

                              Oh! I must be confusing it with some other site that I used to read (many years ago) and now I have no idea what it was actually called. They wrote very accessible summaries and analysis of “important” computing papers from the past. Sorry for the misdirection… now I’m deeply curious what that site was.

                          2. 2

                            This unironically sounds like a job for an LLM.

                          3. 2

                            I’d argue that the overwhelming majority of developer won’t gain much by reading jargon ridden paper that contains nothing interesting because they were written mostly to meet a target amount of published work by their author.

                            I say this because most dev work on the “variation of the same CRUD app in new space” mentioned the article. In such a context, one will get a much better bang for his buck by learning deeply about database, the Linux Kernel, the JVM GC, transaction, concurrency… you know, the stuff we have been building our app with for at least the last 30 years. This this is the knowledge that will allow you to fix the prod issue that occurred at 12:30 AM. Most trouble still occur in those domain, which are deep enough and large enough that you can read about them for decade and still learn new stuff.

                            Remember that reading academic does not occur in a vacuum: the time you spend reading them is time you are not reading something else. Personally I find that they are far from the most profitable investment a regular developer could make. Sure, if you want to change the world, to bring software to whole new height, go read them. Its just that changing the world is something that only a few achieve among many who try.

                            1. 1

                              While I agree with the sentiment, many academic papers do not provide the added context, such as the code they used to generate their results / illustrate the ideas, so there is little opportunity to verify the ideas, which would make the process far more valuable, and fun.

                              1. 2

                                arxiv.org has a feature at the bottom called “labs” and there is a tab “code, data, media” See this article as an example: https://arxiv.org/abs/2010.00029 (I just picked this article at random)

                              2. 1

                                I really love the idea of being in tune with the cutting edge, and have a huge admiration for academical Computer Science, but in reality you don’t really gain much from it as a practicing programmer.

                                Most papers have an implicit background to them, so you need to read the references and the references or the references to really get why it’s worth the time. Plus many don’t provide any inkling of how what they describe would be implemented, or why it is significant, perhaps because both of those are already obvious to their intended audiences.