1.  

    These values reflect a deeper sense within me: that software can be permanent — that software’s unique duality as both information and machine afford a timeless perfection and utility that stand apart from other human endeavor. In this regard, I have believed (and continue to believe) that we are living in a Golden Age of software, one that will produce artifacts that will endure for generations.

    I wish I shared that enthusiasm. A lot of software is accidentally permanent. We should rewrite more.

    1.  

      I wish I shared that enthusiasm. I feel like I’m stuck up to my

      armpits in vendored flotsam, flooded by sloppy abstractions hastily implemented

      everyday. While filled with tremendous amounts of coffee the very thought of being asked by my boss if I’m done yet brings me out in a cold sweat.

    1. 9

      Writing tests can prevent problems before they happen.

      1. 4

        I enjoyed reading this - though I read it because your one sentence summary gave me the “but waaait cleanroom” reaction.

        Another summary is “reflecting on your code, mechanically or not, improves quality”.

      1. 1

        Reads like it mainly applies to things that are insanely complicated. People writing about Pascal’s and such might be doing less work on this stuff.

        1. 4

          That’s my sense also. I don’t think many people appreciate how incredibly complex C++ is. Imagine, for instance, designing this into a language non-reluctantly.

          Scott’s career has been largely about digging into these issues and creating guidance for C++ programmers who don’t have the time to spelunk. It’s hard for me to think of an other language where the ‘errata evaluation problem’ would come up. For instance, despite how feature-rich C# has become, answers to questions are straight-forward.

        1. 10

          I posted my view on this here. I know it might be perceived as an official stance of some sort since I was only person speaking for it when I wrote it. I’ve also been cross-posting between the sites knowing there was a rough history. So, I took care to describe the benefits of our site, note the history, note that many of us like both sites, some don’t, and highlight differentiators. I prefer the Hacker News and Lobsters communities coexist peacefully wherever possible since I see them as different things both of great value.

          I’m sure it will come up again there or on another forum. Although I’m mostly set on that stance, I’ll still read any feedback in case there’s a better way of presenting that or if I should’ve added/removed something. I just want it to be accurate historically and currently so people researching aren’t needlessly surprised by anything. They get the picture immediately. I highlight the positives. Then, readers do what they want from there: enjoy it, ignore it, whatever. We’ll still be over here enjoying great content and discussions regardless. :)

          Edit: Saw pushcx in comments as “Harkins” after I submitted it. So, there were two of us with one an actual official, err admin, of the site. ;)

          1. 11

            Personally, I find hacker news to be stressful. Every time I read it I feel like there is a swarm of conflicting arguments. My biggest concern would be the hacker-newsification™ of lobsters. Peaceful coexistence works for me.

            1. 9

              Apologies if this is not the thread, but I did something of a controlled experiment on my “undefined behavior” post a week and a half ago, posting it to lobste.rs first, then waiting for it to be posted to HN and Reddit (rust, programming, and C_programming). The best discussion was /r/rust, then lobste.rs, then HN. The other Reddit threads were quite bad. This is a topic which has often quite contentious discussions, there’s a large contingent of people with thin knowledge but strongly held opinions. I deliberately wrote it in a non-provocative style, most especially circulating it among experts and toning down statements which they identified as controversial or confusing before publishing.

              That experience will directly influence my posting patterns. When I write interesting blogs, I’ll post them here first (I have two in the pipeline: “a few of my favorite sigmoids” and “IIR filters can be evaluated in parallel”) and wait for them to be posted other places or not. Obviously, my Rust stuff will go on /r/rust first, that’s something of a home community for me (it’s so far the only place my leaving Google has been noted).

              1. 3

                This is a topic which is often quite contentious discussions

                I’m also going to add to your interesting study that this is a topic that people have argued repeatedly often not reaching an agreement. I didn’t say much about it for that reason. I bet many civil or insightful people would avoid discussions on topics that were just worn out a lot on a forum. Undefined behavior, using C, and so on are examples of stuff we’ve debated a lot here. So, keep that in mind as you’re trying to mentally model the response. I think that variable also applies to all the communities on such topics.

                1. 3

                  I’m also going to add to your interesting study that this is a topic that people have argued repeatedly often not reaching an agreement.

                  Agreed. I think that makes it a useful metric of a community, how people respond to such topics. I also think some people deliberately write things designed to stir up controversy. Likely HN is at much greater risk just because the community is so much larger, and of course people there routinely take the bait.

              2. 6

                I asked around a year ago if they could update my username from ‘Harkins’, which is not even my last name anymore, to the handle I use everywhere now, but it was technically hairy. And, yeah, +1 on peaceful coexistence. Not just because they’re 100x bigger and could sit on us, but because online communities aren’t a zero-sum game, let them do their thing their way.

                1. 6

                  The thing I like about lobste.rs is that it is focused directly on technology. HN ’s problem is its wider scope: tech + startups and entrepreneurship. In that scope, a vast array of social topics become fair game for discussion. A sizable subset mutate into ‘dumpster fire’ threads.

                  1. 1

                    From your comment:

                    We also did a mass invite not long ago that changed things up a bit.

                    When was this happening?

                    1. 2

                      Probably referring to invite pledge drive. Here’s where it started and ended.

                  1. 6

                    Really important point here is that most engineers are only at a job for a couple of years.

                    That being the case, it is extremely unlikely that we as engineers will be around as issues of scale come up, so we might as well err on the side of making our jobs as easy as possible.

                    1. 4

                      I’m not sure whether to invoke The Golden Rule right now or karma.

                      1. 10

                        I would much rather work on a code base that didn’t preemptively try to scale than one that has non working scaling code.

                        1. 2

                          I agree here. Having worked on a lot of old-ish codebases, application drift and environment drift tend to cause whatever the original author thought would be the biggest scale issues to be non-issues within a few years.

                          For instance, we have a handful of big hulking java codebases whose bulk is mostly related to making it possible to switch between Oracle and DB2. We haven’t used DB2 for anything in 15 years, and none of that code was useful for making it possible to switch to postgres. We have thousands of lines of code intended to compensate for unreliability in ancient DB2 installs, which actually make reliability worse against Oracle or Postgres while making all the processes much slower. The people who designed this left ten years ago, and for the past 5+ years the main bottleneck has been due to premature optimizations that were ultimately ill-considered (like using hadoop for mostly-linear processes that would be faster if done in a shell script, using expensive unreliable third-party services for what amounts to tokenization and ultimately doing our own tokenization most of the time when the third-party service times out or crashes, or adding unnecessary extra steps that require changing the format and location of tens of terabytes of data based entirely on the expectation of nonexistent performance gains for features that were never implemented or seriously planned).

                          Premature scaling is premature optimization. Just like optimization, it makes more sense to write something straightforwardly, identify actual bottlenecks, and rearchitect and rewrite to counteract them. If rewriting a project you’ve already written is hard, that indicates that you’re using the wrong tools (or that you’ve created the wrong design for the toolset you’re using).

                        2. 2

                          Upvotes are always appreciated. ;)

                          That said, again, making our lives as existing engineers easier is also likely going to help future generations of engineers on the codebase. It’s very hard to execute a Great Leap Forward even if everyone is around to maintain it.

                      1. 4

                        I think they are approaching the problem at too abstract a level. And, no, I’m not being facetious.

                        1. 2

                          It’s interesting to notice that Smalltalk had this right from the start. Constructors are a convention and they are methods on the class rather than the instance. That might not seem important, but having this “factory” level as a syntactically simple convention allows clean separation of the work needed to create an instance, work that could fail on its own, and the normal invariants that you’d have for an object when it is ready to use.

                          1. 4

                            Like the author, I’ve been at this a long time. Most recently, I’ve been working with people who are practicing ‘mob programming.’ I like it but I have mixed feelings about it. As a technical coach, it’s great. When you have a design discussion, everyone is there. Learning and knowledge diffusion are maximized - but I can’t help feeling that it really is slower and not ideal for all circumstances. Distributed work has been around for decades but the trend to make it “the way we do things” at companies is recent. Now people are saying it’s not just convenient, it’s better.

                            It’s interesting that we have these two diametrically opposed ways of working and advocates of each don’t just say “Hey, I like to work this way.” They proselytize and make the pitch that their way of working is best. In fairness, the author is saying something more nuanced, but yes, it is a pitch. I agree with him when he says context matters. It’s ok to admit that preference is a part of the context.

                            1. 2

                              I did this once when designing an in-house programming language. It was a DSL for statisticians and it compiled down to C++. To get the users working quickly, I changed the parser so that anything between double ‘@’ pairs was emitted directly as C++. It was a good way to get people working as the language was being developed.

                              1. 3

                                I wonder whether the introduced bugs or any pattern they form are detectable? If they are, attackers would move on to other targets rather than get trapped in ‘flypaper.’ Making attackers believe that the bugs are exploitable would be the real win. It would be like the tactic of keeping the telemarketer on the line to keep them from calling others.

                                1. 3

                                  Making attackers believe that the bugs are exploitable would be the real win.

                                  That’s a really, common strategy called honeypot systems. Some even fake entire networks.

                                  1. 2

                                    I believe the initial assumption is that people treat large classes of bugs, like “the program crashes on invalid input”, as promising exploit candidates, in part because there is tooling to find those kinds of bugs (fuzzers and such). So you can maybe make that search harder if you inject a buch of non-exploitable bugs for each of those common categories, so that fuzzers turn up far too many false positives. But yeah, then you have the usual arms race: can people just narrow their heuristics to exclude your fake bugs? There’s a small discussion of that from one of the authors on Twitter.

                                  1. 2

                                    No need to speculate on control theory or continuous models in software: they’re heavily used in embedded field in industrial control, aerospace, and so on. Certainly be beneficial for more developers to look into that stuff, though.

                                    1. 3

                                      Yes, I agree, that they are used to create software that controls or models things in the world. I was getting at the point that those tools can be used to control or model software systems. We do this to some degree in site reliability but likely there’s more we can borrow.

                                      1. 4

                                        ActivityPub strikes me as the invention of people who believe that the internet = HTTP, and who know about JSON but not RFC822.

                                        Some of the example message bodies just look like JSON-ized SMTP headers, “inReplyTo” etc. It looks like it has a MIME-inspired “mediaType” attribute too, but does it allow only one media type per message?

                                        Can someone who is more familiar with ActivityPub give me the sales pitch about why existing protocols don’t suffice?

                                        1. 6

                                          RFC822 is ASCII only to begin with one of the biggest limitations of email related “standards”.

                                          Some 6.5 billion people around the globe use non-ascii charecters, and old standards only have layers of hacks upon them to support their usecases to some extent.

                                          Why not create new standards from the ground up for the current usecases? I’m not interested in ActivityPub curently, but I have some experience with email and related technologies, and it badly needs a redesign. It won’t happen as none of the parties capable to organise it is interested in it.

                                          1. 4

                                            My uninformed guess is that with the slow decline of email, there are more & better JSON parsers than there are MIME or email parsers. I would have made the same choice, but my reason would have revolved around JSON’s ability to store structured data, for future flexibility.

                                            1. 2

                                              HTTP Headers are the same format like MIME headers, browsers already have everything one would need for mail. Multipart documents (email attachments) are the same format like HTTP file uploads via form. There is a number of headers both share.

                                              1. 1

                                                I think it comes down to tooling. Protocol A could be 10x as widely deployed as protocol B, but if protocol B has better libraries, I’ll give that more weight in my decision of which to use. I had to assemble a multipart MIME message for work a few weeks ago, and everything about the experience was inferior to “create a data structure and convert it to JSON”.

                                                Coders are likely to pick the easiest path, if everything else is roughly equal.

                                            2. 1

                                              No reason, really. It’s a marketing effort, mostly.

                                              1. 1

                                                SMTP is forever tainted by spam. ISPs like to block ports, spam filters like to eat mail from new unknown servers, etc.

                                                Giving a pitch for Webmention instead of ActivityPub: Webmention requires the sender to publish an html page that actually links to the target URL. You can be stricter and require a valid microformats2 reply/like/repost/bookmark. That already stops old school pingback spam. For stronger protection, there are clever schemes based on “this non-spam domain you linked to has linked to me”.

                                            1. 14

                                              I’ve been reading a lot of Nancy Leveson’s work and she provided had an amazing explanation for why software engineering is so different from “conventional” engineering. In f.ex mechanical engineering, the main danger is component failure: Something breaks and cascades through the machine. In software engineering, the main danger is emergence: the combination of multiple interacting pieces, all working perfectly, leads to a global problem.

                                              It’s not a “we’re more incompetent than the REAL software engineers”. She studied the designers of missile systems, power plants, and aircraft, all places that take software engineering extremely seriously. But they’re all still buggy for emergence reasons.

                                              1. 9

                                                It sure feels like the emergence is the consequence of the sheer scale.

                                                I came across this tweet recently https://twitter.com/nikitonsky/status/1014411340088213504 Bet that missile systems, power plants, and aircraft all have less code than many relatively simple desktop apps.

                                                1. 2

                                                  That’s interesting. A quick search indicates that the F-35, which has had numerous delays and reliability issues (I read somewhere that pilots have to reboot one of the onboard computers every 10 minutes or so) has over 8 million lines of code.

                                                2. 2

                                                  It’s true. I don’t think it counters the point, though. How many of those systems are designed with integration patterns or analyses that ensure the individual components work together properly? I doubt many. The few I’ve seen came out of the correct-by-construction approaches. Even they usually have simplified mechanisms for the integration that make it easier to analyze the system. Many real-world systems use unnecessarily, complicated forms of integration from how they couple modules up to the build systems they use.

                                                  I think emergence will have a mix of intrinsic and accidental complexity as usual. I think many failures are caused by accidental, though.

                                                  1. 1

                                                    Is there a good link to her argument?

                                                    1. 2

                                                      I’m basing a lot of this off her free online book engineering a safer future. She also has a seminar on it here: https://youtu.be/8bzWvII9OD4

                                                  1. 1

                                                    Joe Armstrong would say that we should.

                                                    1. 3

                                                      Does anyone know any more about this? I’ve never heard of it and it seems very new, but there is already a BallerinaCon in July? Looks like it’s owned by WSO2 who I’ve never heard of before either.

                                                      1. 3

                                                        It has been about 3 years in development but we really started talking about it earlier this year. The origins indeed have been in WSO2’s efforts in the integration space (WSO2 is an open-source integration company and had a research project on code-first approach to integration). Ballerina is an open-source project - at this moment has 224 contributors.

                                                        It is getting a lot of interest in the microservices and cloud-native (CNCF) space because it supports all the modern data formats and protocols (HTTP, WebSockets, gRPC, etc.), has native Docker and Kubernetes integration (build directly into a Docker image and K8S YAMLs), is type-safe, compiled, has parallel programming and distributed constructs baked in, etc.

                                                        You can see lots of language examples in Ballerina by Example and Ballerina Guides.

                                                        1. 2

                                                          I actually posted this hoping someone would have more info. The language looks interesting and far along to be under the radar.

                                                          1. 1

                                                            The company seems to be based in Sri Lanka. It is nice to see cool tech coming from countries like that.

                                                            1. 1

                                                              The company seems to be based in Sri Lanka. It is nice to see cool tech coming from countries like that.

                                                              The project has non-WSO2 contributors as well, and WSO2 has also offices in Mountain View, New York, Pao Paolo, London, and Sydney, but indeed Colombo (Sri Lanka) is the biggest office so at the moment my guess would be that Ballerina is 90% from Sri Lanka - which indeed is a fantastic place! :)

                                                          1. 5

                                                            The author mentions that ASCII-friendly APL successors (see J) “are all ugly far beyond the possibility of public success.” While I don’t necessarily agree, I feel like ligature fonts would be a perfect fit for a language like J. It could be used to map verbs onto their APL equivalents, and just make things look a bit more cohesive.

                                                            1. 7

                                                              J is beautiful and, in terms of semantics, is even more elegant than APL. The notation is its primary drawback, to me, for two reasons: there is no longer a one-to-one mapping of action to symbol (because some symbols are digraphs or semi-trigraphs); and because the symbols used already have well-known meanings, causing cognitive burden when switching between the J and the everywhere-else meaning.

                                                              Also:

                                                              This is the Game of Life in APL

                                                              I love APL but I swear if all you read is pop-CS articles about APL you’d think it’s Life: The Language

                                                              1. 3

                                                                Would you have some recommendation of array language snippets that are more representative of the things people end up writing?

                                                                As someone extremely tired of seeing fibonacci examples for functional languages, I’m very interested in knowing what real APL looks like

                                                                1. 2

                                                                  The Co-dfns compiler (https://github.com/Co-dfns/Co-dfns) is an APL-to-C++ compiler written in APL.

                                                                  GNU APL has a pretty nice community page at https://www.gnu.org/software/apl/Community.html where they list some APL projects (some written in APL and others in other languages).

                                                                  J has an extensive standard library and a complete relational database written in J, all at https://jsoftware.com

                                                                  Array languages get the most use today in finance, I believe. The K language from Kx Systems (and the Q query language strongly related to it) are widely used there and have a free-as-in-beer version available with some documentation.

                                                                  (I don’t remember who said it, but the statement “every time you buy stock you’re using K” is probably a reasonably true statement.)

                                                              1. 1

                                                                Seems like he had the answer in the first few paragraphs: Ideally, the people determining the scope and timing of the work should be the people doing the work. After that, he went elsewhere.

                                                                1. 12

                                                                  I thought it would actually be about std::optional, not workspace issues that have nothing to do with the problem at hand.

                                                                  TL;DR: keep your toolchain up to date if you want to use recent language features.

                                                                  1. 3

                                                                    yeah. I suspect better article naming would be better at not leaving people feel like they kept on expecting the article to go somewhere it didn’t.

                                                                    1. 9

                                                                      I think it’s funny because the reader’s experience parallels the author’s experience of wanting to get someplace.

                                                                      1. 4

                                                                        Somebody gets me! :)

                                                                      2. 2

                                                                        Sorry folks :(. But std::optional works as one expects - you can write functions to accept std::optional and you just check early on if it evaluates to true and just return empty as needed, so you can chain functions neatly.

                                                                        Now, if only we could have pattern matching …

                                                                        1. 3

                                                                          I think the consensus of languages with options and pattern matching is “don’t use pattern matching, use combinators”.

                                                                          1. 4

                                                                            Hmm as a full-time Haskeller “don’t use pattern matching” is news to me. Do you mean “don’t use pattern matching for fundamental vocabulary types like Maybe or Either? In which case it’s a reasonable guideline. For types representing your business domain, pattern matching is perfectly good practice. IMHO exhaustiveness checking of pattern matching is an indispensable feature for modelling your domain with types.

                                                                            1. 1

                                                                              Do you mean “don’t use pattern matching for fundamental vocabulary types like Maybe or Either?

                                                                              Yes.

                                                                            2. 3

                                                                              Consensus, really? I’m a big fan of combinators, but I’ll still match on option types sometimes if I think it looks clearer.

                                                                              1. 2

                                                                                Ooh, this is interesting to me - can you expand on this (or point me to some writeups)? Thanks!

                                                                            3. 2

                                                                              Agreed. I read all the way down and nothing significant about std::optional.

                                                                              I thought it was going to be some sort of piece about how using std::optional could lead to yak shaving or something :(

                                                                          1. 2

                                                                            It looks similar to ideas I have as I work on my keywordless language [1]. A simple example would be:

                                                                            ? t < v : ^ ERANGE , v;
                                                                            

                                                                            (where ? is IF, : is THEN and ^ is RETURN). A more complex example is:

                                                                            {? err,c = getc()
                                                                               == EOF , _           : ^ 0   , v;
                                                                               != 0   , _           : ^ err , v;
                                                                               _      , _           : { ungetc(c) ; ^ 0 , v; }
                                                                               _      , is_digit(c) : n = c - '0';
                                                                               _      , is_upper(c) : n = c - 'A' + 10;
                                                                               _      , is_lower(c) : n = c - 'a' + 10;
                                                                            }
                                                                            

                                                                            (where _ is “don’t care” placeholder). Internally, the compiler will [2] re-order the tests from “most-specific” to “least-specific” (so the _ , _ : bit is like ELSE). Also, here, getc() returns two values [3], both of which are checked. I do not have exceptions because I’m not fond of exceptions [5] so I don’t have syntax for it.

                                                                            [1] Based off an idea I read about in the mid-80s [4].

                                                                            [2] I’m still playing around with syntax.

                                                                            [3] I had a hard time moving from assembly to C, simply because I could not return multiple values easily.

                                                                            [4] It’s a long term PONARV of mine.

                                                                            [5] It’s a dynamic GOTO and abused way too much in my opinion.

                                                                            1. 1

                                                                              Very nice. Re [2], does that mean that the sequence of the checks in this construct really is immaterial?

                                                                              1. 1

                                                                                Non-existent. I’m still working (even after all these years) on syntax. It was only after I posed the above did I realize that trying to go from “most-specific” to “least-specific” is problematic in the above example. Of these two:

                                                                                == EOF , _
                                                                                != 0 , _
                                                                                

                                                                                Which one is more specific? It’s for these reasons (and some more) that this is taking a long time.

                                                                            1. 2

                                                                              This article makes me think about the difference between constructs where we have to impose an order on the checks and ones where we don’t. The latter seems nicer but they would lead us to drop ‘else’ and be explicit about the full decision space.