1. 9

    While suckless as a way to build software is definitely interesting (and a lot of the benefits of suckless come from disregarding edge cases, internationalization and accessibility), I’d argue that suckless as a project is something one should handle really carefully.

    They’re doing literal torch hikes through southern Germany at their conferences, fighting online against “cultural marxism”, and their mail server has the hostname “Wolfsschanze” (see: https://twitter.com/pid_eins/status/1113738766471057408)

    I recommend reading this thread (with a suckless.org developer with enabled hat speaking officially) and looking at this photo from one of their conferences.

    1. 18

      I was subscribed to the suckless mailing list for a long time (though no longer, simply out of disinterest), and never had the impression I was dealing with a group of extremists (other than a rather extreme take on software). I don’t recall any political discussion off-hand, and would certainly have unsubscribed if people started ranting about “cultural Marxism” and the like.

      I read the Lobsters thread you linked and there are many things I personally don’t agree with, but I also find it’s a lot more nuanced than what you’re suggesting (specifically, there was a lot of confusion what was even intended with “Cultural Marxism”). I saw that on HN you (or someone else?) linked to an old tweet of yours that screenshotted just the initial “Cultural Marxism” mention of FRIGN, and I think that’s unfairly out of context. That’s not a defence of the contents if his posts, only a defence of treating people with fairly and with kindness.

      I find putting the picture of the torches next to literal Nazis and the “Unite the Right” rally incredibly tasteless and offensive. Note the suckless event happened before the Charlottesville march (not that it really matters). [edit: incorrect, see follow-up]. I’ve done torch hikes – they’re actually used to celebrate the end of Nazi occupation in my home town every year and participated regularly. I’ve also done them with scouts just for the fun of it. Maybe some day someone will dig up a picture of that too and put it next to a bunch of Nazis to prove a point… I’m very disappointed anyone would try to make a point like that, here or elsewhere. This part of your post in particular is really bad in many ways IMHO; it’s really not acceptable to just sling around grave insinuations like that based on a friggin’ contextless photo of what is almost certainly just a harmless social event.

      The mail server belongs to an individual (@FRIGN here). I agree it’s in very bad taste, offensive, and that Poettering was completely right in calling that out, but it’s hardly proof that “they’re a bunch of Nazis”. I find the jump from “edgy hostname” to “literal neo-Nazis” a bit of a leap.


      I doubted for a long time if I should post this reply as it has the potential to spark a long heated discussion, but I find public casual comparisons to Nazis in particular serious enough to warrant something of a rebuttal.

      1. 5

        Note the suckless event happened before the Charlottesville march (not that it really matters).

        I just want to comment on this one factual point, according to the suckless website this event happened in September 2017, just a couple of weeks after Charlottesville.

        https://suckless.org/conferences/2017/

        I do think the proximity in time to the Unite the Right rally is important, especially given the insistence that they were just enacting a German cultural practice.

        1. 6

          Oops, I checked the website and I misread that date as being on “2017-01-03”, instead of “2017-09-(01-03)”. How silly 😅🤦‍♂️

          I’m not sure it matters all that much though; it still seems incredibly tenuous at best. This happened on the other side of the world and I’m not sure if the entire world should tip-toe around sensitive topics in the United States. Were these people even aware of Charlottesville? And to what degree? Me, personally, I mostly stopped following US news since the 2016 election as I find it emotionally draining and serving little purpose as it’s not in my power to do something about anyway.

          Either way, I’d sure like to see some more evidence exactly because I take it serious: you just don’t go around insinuating people of such serious things with such slim “surely it can’t be coincidence…” type of stuff.

          1. 22

            I was at the torch hike and hadn’t even heard of the Charlottesville marches then. When I heard the accusation that we in some way celebrated it, which would make no sense in the context of a software conference, I first had to look up what they were.

            The thing is, Americans tend to overestimate the importance of domestic events like the Charlottesville marches and think that nothing happens in the whole world and, e.g., we Germans are just sitting at home and waiting for something to happen in the USA to witness it.

            The truth, and I think everyone would agree that this also makes much more sense, is that torch hikes are perfectly normal in Germany. I have an understanding for this cultural misunderstanding, and I’ve been guilty of those, as well, but it doesn’t help when one continues to spread this nonsense that this torch hike was some political event every time suckless is discussed here.

            To give an example for how normal torch hikes in Germany are, there is a so-called Sommertagszug in the Kurpfalz which also involves torch hikes at night. They are also offered by tourist organizations, e.g. Breitbach Klamm.

            1.  

              As an American, thanks for sharing your perspective. It makes me wonder if the Internet, and particularly social media, make it too easy to carelessly make connections between things that should remain disconnected. Maybe Facebook’s stated mission of making the world more connected (whether or not that’s their real mission) isn’t a totally good thing.

              1.  

                It definitely comes at a cost. Still, as I could see from my own experience, after a few years one gets more careful with culture-relative judgements. There are still many things Americans do that I don’t quite understand or find interesting.

                To give an example, I found out a few years ago that the German “mhm” (i.e. the expression to acknowledge you are listening to someone while he speaks) is often interpreted by Americans as a “huh?”. You could imagine how much confusion that caused.

                Cultural differences are valuable, though, and I would not want to miss them, even if they become troublesome. I can imagine an American coming to Germany to experience a torch hike and liking it.

              2.  

                What’s with the mail server host name though? Do you think that’s fine?

                1.  

                  Going on a torchlit hike at night sounds fun to me in the abstract, and also like the sort of activity that could hardly be unique to any one place, time, or culture. For ages before the invention of electric flashlights, how else were human beings supposed to light their way when walking around at night, wherever in the world they happened to be? I was unaware that some people associated the practice of going on torchlit hikes with specifically the NSDAP (or maybe just going on a torchlit hike while being an ethnic German??) until I saw people mentioning it in the context of suckless. Even if it’s true that the historical Nazis practiced torchlit hikes (which I assume is true, because I think it would be very easy for any group in human history to do so), I don’t think that confers any obligation on people alive today to refrain from it, any more so than Adolf Hitler’s famous vegetarianism confers any obligation on people today not to be vegetarians.

                  1.  

                    I agree. I’m pretty well read on the topic, including having read Shirer’s “Rise and Fall of the Third Reich,” and I hadn’t heard about the association between torchlit hikes and Nazis before it was brought up in the context of suckless either. If I’m actually educated on the topic and still didn’t know about it, how could I really expect others to know about the association?

                    Personally, a torchlit hike sounds like a blast to me. If the opportunity presented itself to me, I would absolutely participate.

                    I agree with others in this thread that people are generally way too quick to bring up Nazi associations. I like to think I’m not naive about it either, since there are trolls and Nazis online that like to play these kinds of games. But I personally expect some pretty firm evidence before I’m willing to entertain Nazi accusations seriously. It’s a pretty serious thing to say.

          2. 17

            The topic pops up here and there, and one should always consider that Lennart Poettering used this bait to easily escape from a discussion about his software that I personally think should take place. Suckless is not directly a coherent group and more of a group of like-minded individuals, so I’m careful to speak “for” the project even though I’m first chair of the legal entity suckless.org e.V..

            What I can say is that we are probably one of the very very few software projects left that do not politicize. We have members from all political spectrums, but make it work, because we only discuss software and nothing else. Those disagreeing with us or unaccustomed to non-political software projects try to put us into a corner, but it makes no sense when it is used to disregard the suckless philosophy itself, which is non-political.

            Torch hikes are nothing unusual in Germany and there was no political intent behind it. Though I do understand now that it might send a different message worldwide, I expect more cultural sensibility from every observer before “calling us out” for allegedly re-enacting nazism or celebrating a Charlottesville march, which is a ridiculous assessment.

            1. 14

              One should always consider that Lennart Poettering used this bait to easily escape from a discussion about his software that I personally think should take place.

              Perhaps, but I don’t think calling out getting emails from a wolfsschanze host is that unreasonable to be honest; as I mentioned in my other post I’m not going to attach far-fetched conclusions to it but I do find it in pretty bad taste. At any rate, to ask it plainly, what’s the deal that?

              1.  

                There is no such thing as “non-political”, because we live in a society with power imbalances. Therefore not taking an explicit political stance, translates to implicitly supporting the status quo, whatever that status quo is in a given society at a given time. You’ll find that people in underrepresented demographics will largely avoid your project as a result, regardless of the political views among members of your project.

                If supporting the status quo is what you intend to do, then that is one thing. But please stop presenting it as “non-political”, because that is simply not the reality of it. It only looks non-political if you yourself are in a position where the status quo benefits you. Which I am also - so this is not a personal accusation, to be clear. But it is something you need to be aware of.

                1. 10

                  not taking an explicit political stance, translates to implicitly supporting the status quo

                  No no no, I cannot agree with that. Let’s take an example. I’m working on a crypto library, that on many aspects is very close to the Suckless ideals: it’s in C, it’s small, it’s easy to integrate into other projects… One of the motivations for it was to fight unnecessary complexity. A fairly political goal if you ask me: if software becomes as simple as I think it can (and should) be, the changes could be felt throughout the global economy itself.

                  My project also has an implicit endorsement of the status quo: it is written in English, and I have no intention to translate the documentation, or even the web site to other languages. Not even French, my native language. Sorry, you need to learn English to use my project. That’s kind of an implicit endorsement of US hegemony. Not that I’m very happy about that, but not fighting that fight does make me reinforce the ubiquity of the English language.

                  But there’s no way my project can have a stance on everything. Its stance on many many subjects is really neutral. It does not fight nor reinforce the status quo. Veganism? Patriarchy? White supremacy? I hardly have a community to speak of, there’s just not enough people to warrant something like a code of conduct. That does not mean my project implicitly rejects vegan transgender black women. In fact, I do hope they’ll feel as welcome as anyone else. And right now, I believe being nice to whoever contacts me is enough.

                  1.  

                    I couldn’t have put it better, thanks for sharing your thoughts. I always like to consider the example of Chemistry: In the 19th and 20th century, German scientists were leading in chemistry and most papers were published in German. Chemistry students were more or less forced to learn German to understand these papers, and German became the lingua franca of Chemistry, which has changed to English though.

                    In computer science, English is the lingua franca. I don’t think it’s exclusionary to only offer software documentation and code comments in English.

                  2.  

                    That is a good point and I understand what you mean with that. For our conferences, we actually offer those who are unable to attend due to financial reasons to pay their travel expenses and accomodation for them, which was greatly appreciated especially by younger programmers who often don’t have the means to fund such a travel.

                    Apart from income differences, that might be a deciding factor being unable to attend a conference and meeting like-minded people, I see no other factors that might hinder someone from joining us. You basically only need an internet connection and a computer. The computer doesn’t even need to be that fast, unlike if you, for instance, intended to work with deep learning software.

                    And if you still criticize the conferences for being exclusionary in some way: Most communication takes place on a mailing list and IRC, many people use pseudonyms. Factors like race, country of residence, gender are thus irrelevant and even non-determinable, if you choose to, and the development on mailing lists and IRC is the main way development happens and there’s no need to do anything else to partake or make submissions.

                    So, again, I know what you mean, but suckless is not an example for a project supported by the status quo. Most people disregard suckless as too extreme in terms of software philosophy and conclude that we would also be extreme in other areas of life, but suckless, for me, is software zen, and everyone is welcome to adopt this philosophy.

                    1. 5

                      Factors like race, country of residence, gender are thus irrelevant and even non-determinable, if you choose to, and the development on mailing lists and IRC is the main way development happens and there’s no need to do anything else to partake or make submissions.

                      This is a common point of view among those in privileged demographics. However, it is also a misunderstanding of how underrepresented people in demographics actually choose where to hang around and contribute, and why.

                      Imagine for a moment that you are someone in a demographic who’s frequently a target of harassment. The exact demographic doesn’t matter much - maybe you’re black, or you’re a woman, or you’re transsexual, or whatever else. But for the sake of the example, imagine that you are a woman.

                      Now, there are two different communities for you to choose from:

                      1. A community that says “we don’t police member’s politics, this is purely a tech project”.
                      2. A community that says “we actively welcome women”.

                      Where are you going to feel safer? In the second community - because there, it’s clear that if someone finds out you’re a woman, them harassing you over it isn’t going to be tolerated and the harasser is going to be thrown out.

                      In the first community, you just kind of have to stay quiet about your identity, have everyone assume that you’re a guy, and hope that no-one finds out the truth. If they do - maybe there’s some persistent stalker following you around and posting about you in every community you join - you can basically predict ahead of time that harassment and other discriminatory behaviour is not going to be acted upon, because “people’s own politics are not policed”.

                      In a way, there are parallels here to how gay people are “tolerated” in many countries. It’s “fine so long as you don’t bother me with it”, which effectively means that you cannot speak about it publicly or have a public relationship with someone of the same sex, because then the cover falls away and you are no longer “okay”, because your identity can no longer be ignored. Harassment (and often violence) promptly follows.

                      “Don’t ask, don’t tell” policies like this don’t make for healthy, diverse environments. They make for environments in which the status quo is preserved, and where the only way to be vaguely safe as a minority is to never tell anyone that you don’t fit into that status quo. This is not inclusive, and it absolutely does support the status quo. Those who fall outside of it will silently move on to healthier communities.

                      I would like it if “who you are doesn’t matter, it’s about the project” were the reality, I really would. But that just isn’t how things work by default in a society with power imbalances, and the only way to get there is by actively enforcing it - and that means taking a political stance, one that disavows discriminatory behaviour and harassment.

                      1. 6

                        Now, there are two different communities for you to choose from:

                        1. A community that says “we don’t police member’s politics, this is purely a tech project”.
                        2. A community that says “we actively welcome women”. Where are you going to feel safer?

                        I don’t know how the suckless community is, but I am convinced that, if I had a dime for every company, group or project that claimed to “actively welcome women” or “promote equal opportunity for everyone” or “have a zero tolerance” towards discrimination, sexual harassment or any other one of the multitude of abhorrent behaviours that plague our industry – and then turned out to be cesspools of prejudice and discrimination, I would be so outrageously rich that even thinking about it is embarrassing.

                        (FWIW, in addition to witnessing it enough times that it’s part of why I seriously contemplated switching careers at one point, I have some first-hand experience with some of that: my most useful skill, career-wise, has been an impeccable accent. Slightly Irish-sounding (which lots of folks in the US seem to fetishize for some reason), which I developed purely by accident (I’m from nowhere near Ireland, I’ve never been there, and I am not a native English speaker) and is extremely embarrassing every time I’m talking to someone who has a real Irish accent. I certainly had it easier than my black or hispanic colleagues – most Western managers of various importance in the corporate hierarchy could immediately identify them as worthy of contempt, whereas in my case it could take weeks before they realized I’m not a white expat, just some Eastern European programmer.

                        Edit: in case anyone’s wondering – the reason why I can be so light-hearted about it is that, for better or for worse, this experience has been largely confined to the workplace, after-work drinks, conferences and the like. I got to live with it for like 40 hours a week at most, and never really got a taste of it before well into adulthood. I always had alternatives and always had refuge – I could always put up with it on my own terms, which most people can’t)

                        Coming from a culture whose closet is not devoid of skeletons in this department, either, I certainly agree that the mere act of not discussing race, or gender, or ethnicity is in itself a privilege that not everyone has. And that it’s up to every one of us to actively fight discrimination, and to make the world safer and more inclusive for those whose voices are silenced by intolerance. But I don’t think it’s reasonable to ask people to integrate that in every single thing they do. Even activists don’t fight the good fight 24/7, I don’t think it’s unreasonable that some people choose to do it only to a limited extent, or in a possibly misguided way, as part of their hobby project.

                        1. 6

                          I might’ve been a bit unclear. A don’t-ask-don’t-tell approach can be taken by members, if they prefer (many communities don’t provide that luxury and e.g. require clear-name-contributions), but doesn’t have to be. We just don’t care about genders or other aspects other than your coding skills. I see that you have a different opinion on this, which is cool, but the suckless philosophy does not extend beyond software aspects and I personally (not speaking for the group) don’t see a reason to extend that.

                          1.  
                            1. A community that says “we don’t police member’s politics, this is purely a tech project”.
                            2. A community that says “we actively welcome women”.

                            The two may not be mutually exclusive. Although there’s certainly a huge overlap, there’s a difference between advocating the revocation of women’s right to vote, and actually harassing women in a particular group, be it an open source project or a chess club.

                            A president of a chess club, or a maintainer of an open source project, can hardly be expected to be aware of the political views of the members, no matter how extreme. He could pry, but that would be uncomfortable for many people, and ultimately exclusionary. We could do it anyway, and define the range of acceptable political opinions, and exclude the outliers. We could exclude traditionalists, or we could exclude gay marriage supporters. We could exclude white supremacists, or we could exclude black panthers sympathisers.

                            In my opinion this would be neither ideal nor possible. As long as people stay courteous and focus on working towards whatever common goal the group has, we could actually have, say, gay and homophobic people working together. So we probably want to define a range of acceptable behaviours instead. For instance, revealing your sexual preferences is generally acceptable (unless maybe you’re too loud about this), and revealing your contempt for people who don’t share that preference is generally not.

                            That’s what codes of conduct ultimately do: they don’t talk about the politics one might have outside of the group, they define a range of acceptable behaviours within the group. Yes, that range will tend to filter out people with some particular political opinions. Few white supremacists will follow a black maintainer. But I would think real hard before I make that filter explicit.

                            I’ve seen it done, and it’s not pretty. I’ve heard of someone being disinvited from some conference because of their political beliefs, even though they (allegedly) never let them seep through or ever behaved inappropriately. I have also heard of someone being fired over their sexual practices (at the behest of SJW, ironically). And at the same time, some people who seem to engage in genuinely harmful behaviour (such as straight up sexual harassment) are not excluded. My suspicion? Enforcement goes after the easy targets, instead of going after the important ones.

                            1.  

                              we could actually have, say, gay and homophobic people working together.

                              Honestly, this free speech absolutism is whack and that’s why I’m out.

                              You don’t know what the fuck you’re allowing. I do - you’re allowing someone who literally spreads hate to walk into work, meekly get some shit done, then go home to post on the internet how trans people are all pedophiles and should be killed.

                              Fact is, you can’t divorce your life from politics because where many of us stand, all minorities, live under the continuous threat that we’ll be murdered, denied service, beaten and reviled all because some free speech absolutist like you envisions a future where racists and their victims can work side by side.

                              My community just had their 12th death. Death because people like you continually give deference to allow our killers to bask in their hate speech until one of them spots us and brutally kills us.

                              You enable this. I’m so happy (not) to be the sacrificial lamb for your perverse ideology.

                        2.  

                          On the other hand, I don’t think it’s realistic to expect every project to look in depth at difficult social problems and form some sort of consensus on how to best deal with it.

                          You’ll find that people in underrepresented demographics will largely avoid your project as a result

                          Why would that be the case?

                          1.  

                            On the other hand, I don’t think it’s realistic to expect every project to look in depth at difficult social problems and form some sort of consensus on how to best deal with it.

                            I think that’s entirely reasonable. This is pretty much the basis of community management in general. It doesn’t even need to be done by the core developers, but someone in the community needs to do it, if you want a healthy community.

                            Why would that be the case?

                            Because they know that their safety is not assured in communities that refuse to take an active stance against bigotry of various kinds. I’ve gone into more detail about this in this other subthread.

                          2.  

                            There’s no such thing as “non-political” software projects because any political actor can decide that the way your software project run things is bad and should be made to change. And if you resist this, you find yourself in a political conflict, even if you didn’t want to be.

                        3. 7

                          As an engineer child of social scientists, I’ve concluded that mental models like that are basically what you get when you take an engineering approach to social systems to its logical conclusion without considering people as, well, people. You end up with very efficient, streamlined, rational systems that place no value upon the people who are crushed in the process. It’s a simple, effective solution to the very complicated problem of human society, and it makes the complicated problem simple by saying “the people on the losing side don’t matter”. You can see this approach working efficiently and effectively all throughout human history, usually in the form of mass graves.

                          Everything should be made as simple as possible, but no simpler.

                          1.  

                            Because I can’t be sure which comment you’re replying to (AFAIK there’s no “parent” link for comments here), can you please clarify what you mean by “mental models like that”?

                            1.  

                              Sorry, I was talking about mental models such as the ones described by this comment: https://lobste.rs/s/nf3xgg/i_am_leaving_llvm#c_01mpwm . Essentially “we are not going to worry about equity and equality because it is irrelevant to the problem we are actually trying to solve”. Works fine when the problem you are trying to solve is “design a machine that does a particular thing well”, but ignores lots of ugly externalities when it comes down to the social structures and organizations of the people actually doing the design. Like unfettered free-market capitalism, it sounds ideal in theory and that makes it an appealing position. But my observation has been that it works great for the people already powerful enough or lucky enough to be unaffected by those externalities, and does not actually make the world a better place for anyone else.

                          2.  

                            Extremes are rarely good. There should not be an aura of aggressivity around any project.

                            1.  

                              They’re doing literal torch hikes through southern Germany

                              I have no idea what holding torches might mean in this context. Could you explain, or provide links?

                              1. 7

                                It looks like one of those things Nazis ruin for everyone - https://www.theatlantic.com/politics/archive/2017/08/why-they-parade-by-torchlight/537459/. Whether that is intentional on the part of the suckless folks, is not clear to me.

                                The other top hit I got when googling was a torchlit tourist hike through Partnach Gorge in Garmisch-Partenkirchen. I’ve been to that gorge (not by torchlight) and it’s pretty cool!

                            1. 3

                              Good writeup, but off-topic both for being content marketing for a book and for being personal drama.

                              1. 3

                                What?

                                Surely this is wrong. Does Rust have this issue?

                                1. 14

                                  It’s not really about Rust. What Rust does is insert the deallocation code automatically when the owner goes out of scope, but C or C++ code would need to deallocate an equivalent structure at some point (or they could decide to never deallocate and just let it leak, which is also an option in Rust), and the cost of deallocation isn’t based on the language. Chasing 1,000,000 pointers to free what they’re pointing to is going to be expensive in any language.

                                  1. 4

                                    This still seems weird…like, why is the caller freeing what’s passed in? Is there some additional context (say, this function is the last one being invoked in the lexical scope of the lifetime of the value being passed in) here we’re not being given?

                                    Or is Rust really using copy-on-pass semantics that require it to free the copies made during function invocation?

                                    1. 22

                                      Rust uses move semantics for owned objects, which in this case effectively means “the compiler knows how to figure out where to insert free()/drop() calls for you”. The single-threaded example turns into:

                                      fn get_size(a: HeavyThing) -> usize {
                                          a.size();
                                          drop(a);
                                      }
                                      

                                      and the multi-threaded example is, as described:

                                      fn get_size(a: HeavyThing) -> usize {
                                          let size = a.size();
                                          std::thread::spawn(move || drop(a));
                                          size
                                      }
                                      

                                      This is Rust’s real secret sauce.

                                      1. 16

                                        The caller isn’t freeing it. The caller is moving the value into the callee and gives up ownership. The callee is now the sole owner of the value and drops it implicitly (if it isn’t passed around further or passed back as a return value).

                                        1. 8

                                          icefox and freddyb gave good answers and my Rust knowledge is not the deepest, but I wanted to address this part:

                                          why is the caller freeing what’s passed in?

                                          The main alternative to the “move” semantics when passing an argument to a function is to pass a reference instead, in which case the called function would be “borrowing” the value instead of owning it outright. In that case Rust would not insert the deallocation code at the end of the function because the compiler knows that the caller is not done with the value.

                                          Depending on the purpose of the function and how it fits in with the rest of the code, another option for controlling when deallocation happens could be to pass in a reference and have a caller somewhere up the stack decide exactly when to drop it.

                                          1. 2

                                            The other answers all cover aspects, but I feel they may not be clear enough.

                                            When something is passed directly (not by reference) to a function, it will either be moved or copied (depending on whether the type implements the Copy trait, which is only possible for types which can be freed without chasing pointers). In the case of a move, the data now belongs to the callee function, and when the owner (the formal parameter name) goes out of scope (unless ownership is passed back to the caller in the function return, this would be at the end of the function), the Rust compiler inserts a drop call to free the owned data (calling the type’s Drop trait implementation). In the case of a copy, the same still happens, but the callee is working with a distinct copy of the data, so the caller’s copy of it is still valid and will be dropped when its own owner goes out of scope.

                                        2. 4

                                          Doesn’t everything have this issue? It’s just the old memory management trade-off of “would you rather your performance be bounded by the size of the set of things you’re deallocating, or the size of the live set?”, isn’t it?

                                          You can deterministically deallocate things as their owner goes out of scope, which is sometimes going to involve chasing pointers to a large number of interior allocations at an inconvenient moment (like the vector-of-vectors here), so you might want to move ownership onto a less critical thread and deallocate in the background, as shown here or as in COM or some other C & C++ codebases, or stop allocating from general purpose heap, create a specific arena for your large set of allocations, and deallocate the arena in one go.

                                          Or you can have a GC that does a mark-and-sweep and have the same “spend a lot of time chasing pointers to a large number of allocations“ problem, but at a less predictable time.

                                          Or you can have a GC that does a copy-and-compact, which might let you avoid the interior pointer chasing to a large number of allocations (if there’s nothing like finalizers that need invoking), but now you’ve got the opposite problem in that your performance is bounded by the size of the live set instead of what’s being deallocated.

                                          What would a language that doesn’t have this issue do instead?

                                          1. 2

                                            What would a language that doesn’t have this issue do instead?

                                            Probably this:

                                            stop allocating from general purpose heap, create a specific arena for your large set of allocations, and deallocate the arena in one go.

                                            I think it’s probably true for many applications that the common case is all pointers stored in a collection (transitively):

                                            a) are allocated in the same place, specifically to be placed in that collection

                                            b) are not stored anywhere else

                                            c) have finalisers who are only chasing and freeing other pointers allocated in the same transitive tree rooted in the collection

                                            and so releasing the collection would be quick+easy if this knowledge was given to the language/runtime.

                                            A language which made this easy would do well here.

                                            We’re paying for the cost of the fully general case all the time, when we probably don’t need to.

                                            1. 3

                                              Zig makes this pretty easy. The stdlib data structures, and anything else that needs to allocate, take an allocator parameter, which can be an arena (and one is provided by the stdlib.)

                                              1. 2

                                                Thanks, that’s interesting!

                                                I know the c++ stdlib allows you to specify allocators etc, but I was wondering about a “more natural” way of doing it.

                                                In many languages we anchor the constructor to a type: f = foo.New() or similar. That ctor will need to grab memory, which is generally stack, heap or custom allocator.

                                                I guess I was wondering what it would be like if we did something like alloc through an owning object:

                                                func handleRequest(r Request)
                                                  m = r.New(Map) // allocate a map owned by the request. When that goes, so does the map
                                                  f = m.New(foo) // allocate a foo owned by the map. When 'm' goes, so does f
                                                  m["myfoo"] = f
                                                  ...do stuff...
                                                  return // neither m nor f cleaned up, but when caller kills 'r', they are thrown away
                                                }
                                                

                                                If the natural way to alloc was with an owning object, we’d get arena-allocation by default.

                                                You’d need other ways of allocating to handle exceptional cases, but they could be done via a global or similar. Which seems to make sense, since an allocation which isn’t tied to an owning object (and needs to live beyond a request or equivalent) is global.

                                                I assume C++ and rust (and Zig!) all allow this, but I’m wondering how far we can take “first class ownership, allocation and freeing” to enable the default use of memory pools.

                                        1. 1

                                          I hear this a great deal, having gone through the exact same thing with a smart thermometer (for people meat, not food meat).

                                          More recently, some smart lighting I have is all about this. On end of the spectrum, you have a whole bunch of stuff that is basically just a shenzhen special with a remote and no way of easily hacking on it (like this). Everything more complicated requires The Clown.

                                          Since I have a dedicated infra network for my IoT stuff (since the S in IoT stands for security), I don’t have any access for my devices to Clown Services providers, nor do I have them authenticating against The Clown.

                                          LIFX was almost good enough to deal with this, and would grudgingly let me control my lights over the network without involving The Clown. However, sometime in the last month or two they pushed an update that required that I authenticate with the LIFX Clown in order to use my mobile control app. This was just annoying enough that I went back to using normal house lights, like a barbarian.

                                          I think it’s great to have this stuff available as an upgrade path for users, but like out of the box I shouldn’t need to talk to anybody else to use the damned products. >:(

                                          1. 1

                                            What is “The Clown”?

                                            1. 1

                                              s/Cloud/Clown

                                              Makes reading modern devops stuff more fun.

                                          1. 16

                                            (What follows is a quick guess with incomplete historical knowledge, so feel free to correct me or poke holes in it.)

                                            I think the core truth, perhaps the grand unifying theory of programming if I may be so bold, is that programming is just the art of mapping elements of one dataspace into elements of another dataspace. Elements, data. Transforms, functions. If you want to go up a level still, you treat the space of functions as a dataspace, and recurse until you’re back at the primordial stuff of programming. This is not particularly clever or controversial, I hope.

                                            Late 50s and McCarthy makes a language called Lisp, asking “hey, what if we used the same expression for for like basically everything? would that let us make programs that operate on programs?” This is the beginning of modern metaprogramming.

                                            60s come and Dahl and Nygaard make a language called Simula, asking “hey, what can we do to make writing simulations easier? algol is neat, but allocation is hard so let’s try lexically-scoped garbage collection…and writing simulations with shared props is annoying, but parameterized code isn’t good enough, so maybe let’s invent inheritance and subclassing”. This is the beginnings of objects.

                                            Then, we had Kay et al come along and say “hey, wouldn’t it be neat if programs were really just like cells, sending messages to each other?”, and that got a lot of people interested. This is beginnings of the actor stuff.

                                            We also had some boring software engineering stuff Meyer and folks did around design-by-contract saying “hey, what if we started modeling code as modules that made guarantees around what they consumed, what they emitted, and how they could fail?” This is the beginnings of interfaces, exceptions, and all that.

                                            There was also a fellow named Turner and his language Miranda which said, “hey, wouldn’t it be neat if we lazily evaluated everything? could we just specify everything in terms of types?”. This is the beginnings of “modern” functional programming as you’d see in Haskell or whatever.

                                            And then Bjarne Stroustrup cooked up C++ and everything has been terrible ever since.

                                            In particular, C++ muddled together a bunch of these concepts and when industry got a hold of it and had to scale it out people ignorant of the history of things (because really who’s gonna find a proceeding from the ACM in the early 70s on the history of an obscure Norwegian language from the decade prior) started building their own interpretations and rediscovering weird and bad versions of these ideas.

                                            Other languages got built in reaction to C++, focused not on refining the original concepts that were mutilated in that language but instead on trying to clean up the ugliness that C++ had turned them into. Along the way, massive improvements in computing power meant that focus on developer ergonomics was even more important than any fundamentally good pure principles in a language.

                                            And so, several generations on, we have people freaking out about “OO” (really vulgar OO, a bizarre mutt of several much smaller, much more fundamentally useful ideas) languages and panning them in favor of “functional” languages, when they don’t really understand either.

                                            Somewhat interestingly, perhaps the academic nature of languages like Haskell is what has allowed them to present their case so strongly. The people that work with those languages and steal ideas have a much neater wellspring to draw from when making careful, deliberate hybridizations like Rust.

                                            Then again, C won, Javascript won, and the great unwashed masses in their glee tend to outbreed and ultimately render irrelevant those of “nobler” heritage.

                                            1. 1

                                              And then Bjarne Stroustrup cooked up C++ and everything has been terrible ever since.

                                              Thanks for a good chuckle, this made my morning :)

                                            1. 16

                                              We, um, don’t seem to have a pascal tag. So, there’s that I guess.

                                              1. 4

                                                Wow! I wonder how many games on Steam are written in Pascal.

                                                1. -1

                                                  Why? It is a mature and tried language. It’s not fancy enough. Unlike Rust. /s

                                                  1. 4

                                                    at least now we have the possibility to rewrite it in rust!11

                                                1. 5

                                                  Last time I’ve tried to deal with ThreeJS I had similar problems…part of the frustration is that there are a lot of libraries that kinda assume “of course you’re using a full build pipeline for the frontend!”, when the task at hands merits maybe a <script> tag with an IIFE.

                                                  It’s all so tiresome.

                                                  1. 24

                                                    This is an advertisement for a Kickstarter and does not contain the type of technical content worthy of a Lobsters post.

                                                    1. 7

                                                      It links to these technical places:

                                                      1. 11

                                                        Those would’ve been better submissions!

                                                        1. 4

                                                          These are what I wanted to see, and I’d have been tempted to hide/flag if the content given had lacked any links to further technical detail.

                                                          But since that wasn’t the case, I think there’s value in highlighting the Kickstarter so people can support the project if they’re interested?

                                                          I’m often tempted to write comments like this on “does x belong here?” threads then abandon them to avoid spamming up the discussion further. So, an actual suggestion:

                                                          Perhaps in cases like this where there’s a good reason to link to the PR material, we could encourage people to also add a comment or blurb calling out any relevant links or technical information?

                                                        2. 7

                                                          Agreed! I think this is squarely on-topic. It’s an interesting product that inspired me to read about https://pwnagotchi.ai/ and other associated devices. I learned a lot from reading the reading this page and searching the web for the topics mentioned. The USB stack fuzzing idea is interesting. I think I might try exactly that on a piece of embedded equipment I have lying around.

                                                        3. 5

                                                          It might be. Or it might just be performance art, as the kickstarter in question does not even exist.

                                                          If it ships, I’d sure like to see a link to the promised firmware and hardware source. It’ll be an interesting gadget if it ever becomes real.

                                                          1. 5

                                                            The heart of it seems to be this very impressive beast https://www.ti.com/lit/gpn/cc1101

                                                            Probably a real swine to write drivers for, but as usual, ti supplies a bunch of libraries you could use.

                                                            Moore’s Law has ceased to be “Doubling of CPU speed every 18 months”. Actually, it never was.

                                                            Actually as stated by Moore, it was “Doubling the number of transistors every 12”. That has reach the peak of how much plain CPU speed that can be delivered.

                                                            What is still happening, is more and more powerful and weirder and more exotic chips like this.

                                                            1. 2

                                                              Can you explain what makes this a weird, exotic, impressive beast? Almost all of my deep knowledge relates to server hardware, I haven’t worked with microcontroller-scale things for about a decade.

                                                              1. 9

                                                                Read the data sheet I linked to for the full details, but basically…

                                                                It’s a radio receiver transmitter (transceiver) that frequency hops over 300-348 MHz, 387-464 MHz and 779-928 MHz bands and converts from an RF baseband signal to a stream of samples (i/q’s) and then encode/decode data from that via a bunch of different modem algorithms, plus a bunch of nifty stuff require to make it practical (sync detect, address and crc check, …) in a tiny (2.4mmx2.4mm) package.

                                                                In The Bad Old Days that used to require a lot of fancy rf circuitry, maybe an fpga, certainly a dsp, and a hell of a lot of very fancy maths and very smart code.

                                                                (Source. I do this stuff for a living)

                                                                1. 1

                                                                  Ahh interesting. Yes, this is why I needed an explanation because I don’t do this stuff for a living. But certainly want to learn more. I can’t do much with a data sheet just yet

                                                                  edit: And thank you!

                                                                  1. 2

                                                                    I’m on the software side of the picture so there is always more for me to learn on the RF/hardware side, but I at least know where the various tasks are done on the device I work on.

                                                                    There are a lot of these super smart super tiny chips coming on the market… the down side smart equals complex to drive, and chip tends to mean you get what you get, if the chip doesn’t do what you want…. get another chip (means PCB redesign) and have a massive task of rewriting your drivers.

                                                                    If you ever pull the bluetooth standard and print it out, (you will need to buy a new bookcase), you get an idea of the scale of “tiny,cheap,super smart aka insanely complex” I’m talking about.

                                                                    That said, a lot of things that were very expensive, are becoming available at commodity prices. In some ways dumber, someways less customised, but with 90% of what you want.

                                                                    And that will probably improve in time.

                                                                2. 1

                                                                  yes i’d love to know as well!

                                                                3. 1

                                                                  i guess if it is paired with another ti µc it should be quite easy. i have still an ez430 chronos lying around, which wasn’t too hard to write code for, given the examples etc.

                                                                  1. 1

                                                                    It would be nice if they marked PDF links more visibly.

                                                                4. -1

                                                                  I don’t think so and so you are wrong because what determines posts to be worthy of lobste.rs is the users, and I am a user of lobste.rs :)

                                                                  1. 2

                                                                    I don’t think a flat “you are wrong” is correct when the post has already been spam-flagged over ten times.

                                                                    1. 0

                                                                      It’s just a joke about “x on lobste.rs should only ever be y” purity posts. You know what they say about death, taxes, and arguments about what lobsters “is”…

                                                                1. 1

                                                                  A webdev who still doesn’t know what a server is after 4 years? I find that hard to believe… This might just be an ad for the organization that is being referred to at the bottom of the page.

                                                                  1. 2

                                                                    Nah, RC is just something a lot of alumni tend to plug in their blog layouts. @pushcx does the same and he runs Lobsters.

                                                                  1. 3

                                                                    I was looking at static site generators for my blog (currently hosted on svbtle). Every time I look into it though the learning curve is inane, so many config options and templating systems. I found one I liked even called zola, but the various, nearly undocumented “extra” configs I needed for themes, and all the work in adapting a theme for all use-cases was enough that I just gave up.

                                                                    I came to the conclusion that hand writing html is probably a great solution, since it’s literally a markup and doesn’t need translation- publishing can just be uploading a file.

                                                                    The concern I had though was that keeping an index and a “next”-“previous” index at the top is more annoying to maintain if manual. And those features are desirable.

                                                                    I don’t have anything interesting enough to say anyway, so maybe I should just drop it.

                                                                    1. 1

                                                                      The next/prev stuff is something I had a little trouble with too. If you store the filenames with a date prefix, your build script can generate the appropriate wrapping HTML. Depending on how fancy you want to be, you can then have the post as an iframe or injected into the layout of a template.

                                                                    1. 8

                                                                      Frankly, my first reading of this had me gritting my teeth and hissing, but I couldn’t figure out why. A second read, accompanied by some background research, and I think I’m both less apoplectic as well as more understanding of what set me off.

                                                                      First, something the author points out that I really agree with is her comment on flow:

                                                                      I just make forms. No genius required. I have no right to complain about noise or interruptions.

                                                                      There’s definitely a lot of truth to this, and I think that many developers (myself at times included) puff up their chests and make a mountain out of a molehill when it comes to what we do. This is exacerbated by picking tools that require more concentration than just a <form> element, even though that’d be sufficient. I think this admission of the author is totally worth reflection by all of us.

                                                                      I also have great sympathy about not knowing how to write bash scripts (I hadn’t really done so myself until a few years ago, and I honestly haven’t done it in anger until the last few months) and AWK scripts (I can semi-reliably print out the first column of a line using AWK without cracking open references right now).

                                                                      I don’t think the author is a bad person for writing this. I think that the writing style may account for some of the abrasive flippancy that I interpreted when reading it. I also don’t know pretty much anything about this author, the circumstances of this post, or many other things that might be extenuating circumstances come interpretation time.

                                                                      For somebody starting out in their career or looking to grow, I think there are three things you want to learn from in this post:

                                                                      Have shame! Use that discomfort to identify places to grow.

                                                                      There are mentions in this post of places where the author didn’t know something (say, certain computing concepts) and identifies that “real programmers” know those things. If you are a real programmer, or even if like me you’re just somebody trying to avoid working retail or digging ditches, your privileged position means that you are kinda expected to learn about that stuff. You should feel bad that you are ignorant of things or unskilled, but you don’t have to remain that way! That dovetails neatly with:

                                                                      Be positive and patient with yourself! View your ignorance as a chance to learn new things and your lack of skills as a chance to improve.

                                                                      Nobody is born knowing how AWK or CSS work! That’s a great chance to explore more and learn how to do your job more easily. Not being sure about how to reliably invokessh or scp means that there’s a new skill you can master and accomplish! Life would be really boring if we were stuck with the skills we were born with–programming is a career with a lot of variety and opportunity for exploration and learning! You just have give yourself the time and room to study up on what’s useful, or have a clear articulation of why you’re setting something down.

                                                                      Learn about the history and the philosophy of our field. It’ll help you help yourself and others better!

                                                                      The author mentions not really knowing about Ada Lovelace, who is considered by many to be one of the protoprogrammers (seeing as how she didn’t actually have any of Babbage’s machines to actually run a proposed program). There are other figures, like Grace Hopper (who, with her team, created FLOWMATIC) or John McCarthy, similarly not mentioned. These are people that started with computers much more primitive than anything you’re likely to run across in your career and yet who accomplished a great deal and shaped the world for decades to come! You have everything you need to do the same, and knowing about the people that came before you can provide inspiration and perspective.

                                                                      When the author uses tools like Google, Skype, and other social media, they mention doing so instead of Tor or Signal or other options “real programmers” use. These other tools are often harder to use and have smaller networks, but tend to offer other advantages in terms of being secure or being able to be run without depending on others. Computers have been around for almost a century now, and yet a lot of people still aren’t comfortable taking charge of their own security or technology–those people are beholden to folks that they’ve delegated to. It’s important to understand both why they might make that bargain, and also to be aware of how you can spread information that can give those people the option to emancipate themselves.

                                                                      ~

                                                                      I think there’s a lot of positive discussion we can base off of this article, even if at first glance it raises our hackles.

                                                                      1. 1

                                                                        Flagged as off-topic since it is basically forum drama. It is also mis-tagged, only rant and law being reasonably relevant.

                                                                        1. 9

                                                                          Do you have any examples of libraries in other languages or language features, which really should have implemented a well-known concept (Monoid, Monad, Alternative, whatever), but they fell short because they (probably) didn’t know the concept?

                                                                          This sort of presumptive arrogance is a massive turnoff for me, and I think that it really hampers my ability to take Haskell users or their community seriously.

                                                                          This whole “well-known” idea is glib coming from the community whose running joke is just a monoid in the category of endofunctors–i.e., famously academic and up its own ass.

                                                                          A better way (hell, we could start it in this subthread) they could’ve brought this up would’ve been “What Haskell idioms (Maybe, monads, etc.) do other ecosystems implement almost like we would but then diverged? What idioms do we have that’d still be doable in those ecosystems?”

                                                                          Instead, it comes off as some ignorant elitist prick dismissing the slovenly masses daring to attempt to fumble their way through shipping code in spite of their own ignorance of the magic of Haskell, somehow managing to cobble together pale imitations of algorithms and techniques in their inferior programming tongues.

                                                                          Pandoc and shellcheck are maybe the only Haskell programs I use on a regular basis, but I could get along without them. I couldn’t get along without the flotsam and jetsam of “inferior” languages like C and JS, and I’m kinda amazed that in 2020 people still represent Haskell like this.

                                                                          EDIT: Just to be clear, I know some Haskell folks (@pushcx, @shapr) that are kind and just excited and happy to share their experience without shitting on other languages I use…but they seem to be in the minority.

                                                                          It’s the difference between “Haskell is the one true language why are you settling for anything else?” versus “I really like Haskell and think it’d be a good fit for your problem, want to talk more about it? If not that’s cool my dude, one day I’ll talk you into it. :)”

                                                                          1. 10

                                                                            I totally understand. Do give that community some benefit of doubt though, as this is a link to a thread within the haskell subreddit where the posters feel free to write without having to preface every post/comment with qualifiers as it might get a little tedious if they had to do that on every thread and comment. If we’re not part of that subreddit, in a way, we are peeking in as an outsider and before judging them harshly, we’ll have to read it with their perspective in mind. Hope I’m making sense. I was almost not gonna post here because of the potential for misunderstanding their tone but it was just a really interesting topic and I was wondering if I could get more viewpoints over here. Although now I’m not sure if it was worth it.

                                                                            1. 3

                                                                              Thank you for the caution towards charity and reasonableness!

                                                                            2. 6

                                                                              I like being called out for kindness and excitement.

                                                                            1. 6

                                                                              Many of the aforementioned features surely suck, but there is no such thing as the correctness of a feature[1]. I have the feeling that these people consider these features “incorrect” because they are not implemented the Haskell way.

                                                                              1: I mean, what’s wrong with the fact that Guido decided that sum won’t work with strings? Nothing. This is a choice. It is valid because it doesn’t conflict with existing Python specs. We could also decide that sum returns 123 if called with a File object. This is another (terrible) choice that isn’t wrong either.

                                                                              PS: Well, in the world of formal specs we can surely talk about the correctness of a feature, but I don’t think that “incorrect” in the title of this page has this meaning. I guess it means something like “not Haskell-friendly”.

                                                                              1. 4

                                                                                What’s wrong with sum() not working with strings? Well, the fact that "foo" + "bar" does work and returns "foobar" as one could expect. The fact that str type has a __add__() magic method and that’s supposed to be the standard protocol for “addition” (whatever it is).

                                                                                Another example from Python is True + 2 = 3. This result is useless, and it misses a chance to catch situations that are almost surely programming errors. It does make "foo" + 2" type error instead of arbitrarily deciding a numeric value of "foo" or (more justifiably) returning "foo2".

                                                                                Overall, Python has an incredibly consistent and logical design, but these behaviours violate the conventions it follows nearly everywhere else in its design, and that’s not a good thing.

                                                                                1. 2

                                                                                  Out of curiosity, how would one concatenate a list of strings in Python?

                                                                                  1. 2

                                                                                    "".join(["foo", "bar", "baz"]). Sometimes, OO approach just makes little sense.

                                                                                    1. 1

                                                                                      It’s nicer ergonomics when you want to concat with a non-empty separator. Which, in my experience, is a very common thing to want. But, you can have it your way if you like:

                                                                                      import functools as ft; ft.reduce(lambda l, r: l+r, ['foo', 'bar', 'baz'])

                                                                                      Still, it would be nice if sum were implemented with __add__.

                                                                                      1. 2

                                                                                        Oh, I’m not denying usefulness of that join API. I just find it funny that it’s presented as a method of the separator string. It also makes it somewhat harder to discover.

                                                                                        It could be simply a function of a separator and a string list:

                                                                                        # String.concat ", " ["foo"; "bar"; "baz"] ;;
                                                                                        - : string = "foo, bar, baz"
                                                                                        
                                                                                2. 1

                                                                                  Though some have been brought up in a world of “fair and balanced” and “all sides are equal”, I think in more than 80% of the cases it’s possible to determine which version of a feature is correct without too much effort.

                                                                                  1. 2

                                                                                    This sort of snide dismissal is how we get Javascript in 2020. Make Ecmascript Great Again.

                                                                                    You shouldn’t be so quick to dismiss the habits of languages and programmers you find deplorable.

                                                                                1. 35

                                                                                  I have very mixed feelings about this article. Some parts I agree with:

                                                                                  It’s you, the software engineering community, that is responsible for tools like C++ that look as if they were designed for shooting yourself in the foot.

                                                                                  There is very little impetus to build tools that are tolerant of non-expert programmers (golang being maybe the most famous semi-recent counterexample) without devolving entirely into simple toys (say, Scratch).

                                                                                  Some of you have helped with a first round of code cleanup, which I think is the most constructive attitude you can adopt in the short term. But this is not a sustainable approach for the future.

                                                                                  […] always keeping in mind that scientists are not software engineers, and have neither the time nor the motivation to become software engineers.

                                                                                  Yep, software engineers pitching in to cleanup academic messes after the fact definitely doesn’t work. One of the issues I’ve run into when doing this is that you can totally screw up a refactor in ways that aren’t immediately obvious. Further, honestly, a lot of “best practices” can really hamper the explorative liminality required to do research spikes and feel out a problem.

                                                                                  But then, there’s a lot of disagreement I have too:

                                                                                  The scientists who wrote this horrible code most probably had no training in software engineering, and no funding to hire software engineers.

                                                                                  We expect people doing serious science to have a basic grasp of mathematics and statistics. When they don’t, we make fun of them (that is, when the peer review system works properly). If you’re doing computational models, you damned well should understand how to use your tools properly. No experimental physicist worth I damn that I’ve known couldn’t solder decently well–nobody doing science that relies on computers shouldn’t be expected to know how to program competently and safely.

                                                                                  clear message saying “Unless you are willing to train for many years to become a software engineer yourself, this tool is not for you.”

                                                                                  Where’s the clear messaging in the academic papers saying “Yo, this is something that I can only reproduce on my Cray Roflcluster with the UT Stampede fork of Python 1.337”? Where’re the warnings “Our university PR department once again misrepresented our research in order to keep sucking at the teat of NSF and donors, please don’t discuss this incredibly subtle work you’re probably gonna misrepresent.” Where’s the disclaimer for “This source code was started 40 years ago in F77 and lugged around by the PI, who is now tenured and doesn’t bother to explain things to his lab anymore because they’re smart and should just get it, and it has been manhandled badly by generations of students who have been under constant pressure to publish results they can’t reproduce using techniques they don’t understand on code they don’t have the freedom to change.”?

                                                                                  The core of that research is building and applying the model it implemented by the code, the code itself is merely a means to this end.

                                                                                  This callous disregard for the artifact that other people will use is alarming. Most folks aren’t going to look at your paper with PDEs and sagely scratch their chins and make policy decisions–they’re going to run your models and try to do something with the results. I don’t think it is reasonable to disavow responsibility for how the work is going to be used in the future if you also rely on tax dollars and/or bloated student tuition to fund your adventures.

                                                                                  There’s something deeply wrong with academic research and computing, and this submission just struck me as an attempt to divert attention away from it by harnessing the techlash.

                                                                                  1. 18

                                                                                    I’m someone who’s done their (extremely) fair share of programming work in academia, but outside a CS department: I can guarantee that anyone insisting that the solution was simple and that it’s just “they should have hired real software engineers” has had zero exposure to “real software engineers” trying to write simulation software. Or if they had, it was either in exceptional circumstances, or they didn’t actually pay attention to what happens there.

                                                                                    (This is no different to CS, by the way. The reason why you can’t just hire software engineers and expect they’ll be able to understand magnetohydrodynamics (or epidemiology, or whatever else) is the same reason why you can’t just hire electrical engineers or mechanical engineers and expect them to write a Redis clone worth a damn in less than two years – let alone something better.)

                                                                                    As Dijkstra once remarked, the easiest machine applications are the technical/scientific computations. The programming behind a surprising proportion of simulation software is trivial. By the time they’re done with their freshman year, all CS students know enough programming to write a pretty convincing and useful SPICE clone, for example. (Edit: just to be clear, I’m not talking out of my ass here. For two years I’ve picked the short straw and ended up herding CS first-years through it, and I know from experience that two first-year students can code a basic SPICE clone in a week, most of which is spent on the parser). I haven’t read it in detail but from glossing over it, I think none of the techniques, data structures, algorithms and tools significantly exceed a modest second-year CS/Comp Eng curriculum.

                                                                                    Trouble is, most of the domain-specific knowledge required to understand and implement these models far exceeds a CS/Comp Eng curricula. You think epidemiologists who learned C++ on their own and coded by themselves for 10 years write bad simulation code? Wait ’til you see what software engineers who have had zero exposure to epidemiology can come up with.

                                                                                    “Just enough Python” to write a simple MHD flow simulator is something you can learn in a few afternoons. Just enough electromagnetism understand how to do that is a four-semester course, and the number of people who can teach themselves how to do that is very low. I know a few and I know for a fact that most companies, let alone public universities, can’t afford their services.

                                                                                    This isn’t some scholastic exercise. No one hands you a two-page description of an algorithm for simulating how the flu spreads and says hey, can you please turn this mess of pseudocode into C++, I’m not that good at C++ myself. The luckiest case – which is how most commercial-grade simulation software gets written – is that you get an adnotated paper and a Matlab implementation from whoever developed the model.

                                                                                    (Edit: if you’re lucky, and you’re not always lucky, that person is not an asshole. But if you think translating Matlab into C++ isn’t fun, wait until you have to translate 4,000 lines of uncommented Matlab from someone who doesn’t like talking to software engineers because they’re not real engineers).

                                                                                    However, by the time that happens, the innovation has already happened (i.e. the model has been developed) months before, sometimes years. If you are expected to produce original results – i.e. if you do research – you don’t get a paper by someone else and a Matlab implementation. You get a stack of 80 or so (to begin with) papers on – I’m guessing, in this case, epidemiology, biochemistry, stochastic processes and public health policies – and you’re expected to come up with something better out of them (and, of course, write the code). Yeah, I’m basically describing how you get a PhD.

                                                                                    1. 7

                                                                                      I can guarantee that anyone insisting that the solution was simple and that it’s just “they should have hired real software engineers” has had zero exposure to “real software engineers” trying to write simulation software.

                                                                                      I totally agree with this. That’s also why my argument is “researchers need to learn to write better code” and not “we should hire software engineers to build their code for them”.

                                                                                    2. 13

                                                                                      …no funding to hire software engineers.

                                                                                      Speaking as a grant-funded software engineer working in an academic research lab, it’s amazing what you can get money for if your PI cares about it and actually writes it into grant applications.

                                                                                      My suspicion, and I have zero tangible evidence for this, just a handful of anecdotal experiences, is that labs outside of computer science are hesitant to hire software engineers. It’s better for the PI’s career to bring in a couple more post-docs or PhD students and expect them to magically become software engineers than to hire a “real” one.

                                                                                      Another interesting problem, at least where I work, is that the pay scale for “software engineer” is below market. I’m some kind of “scientist” on paper because that was the only way they could pay the position enough to attract someone out of industry.

                                                                                      1. 5

                                                                                        Speaking as a grant-funded software engineer working in an academic research lab, it’s amazing what you can get money for if your PI cares about it and actually writes it into grant applications.

                                                                                        Oh, totally agree. I’ve made rent a few times by being a consulting software engineer, and it’s always been a pleasure to work with those PIs. Unfortunately, a lot of PIs just frankly seem to have priorities elsewhere.

                                                                                        I’ve heard also that in the US there’s less of a tradition around that, whereas European institutions are better about it. Am unsure about this though.

                                                                                        Also, how to write code that can survive the introduction of tired grad students or energetic undegrads deserves it’s own consideration.

                                                                                        1. 6

                                                                                          Yeah, “Research Software Engineering” is a pretty big thing in the UK at least… https://society-rse.org.

                                                                                          1. 11

                                                                                            It is (I’m an RSE in Oxford). It costs as much within bizarre University economic rituals for a researcher to put (the equivalent of) one of us (full time, but what they usually get is that time shared across a team of people with various software engineering skills and experiences) on a project as it would to hire a postdoc research assistant, and sometimes less. Of course they only do that if they know that they have a problem we can help with, and that we exist.

                                                                                            Our problems at the moment are mostly that people are finding out about us faster than we’re growing our capability to help them. I was on a call today for a project that we couldn’t start before January at the earliest, which is often OK in the usual run of research funding rounds, less OK for spin-out and other commercial projects. We have broken the emergency glass for scheduling Covid-19 related projects by preempting other work, I’ve been on one since March and another was literally a code review and plan for improvement as the linked project got after it was shared. We run about 3 surgery sessions a week on helping researchers understand where to take their software projects, again that only lands with people who know to ask. But if we told more people they could ask, we’d be swamped.

                                                                                            While we’re all wildly in agreement that this project got a lot of unfair context-free hate from the webshits who would gladly disrupt epidemiology, it’s almost certainly the case that a bunch of astrophysicists somewhere are glad the programming community is looking the other way for a bit.

                                                                                            1. 3

                                                                                              I’m an RSE in Oxford

                                                                                              A lot of UK universities don’t have an RSE career track (I’ve been helping work to get one created at Cambridge). It’s quite difficult to bootstrap. Most academics are funded out of grants. The small subset with tenure are funded by the department taking a cut of all grants to maintain a buffer for when they’re not funded on specific ones. Postdocs are all on fixed-term contracts. This is just about okay if you regard postdoc as a position like an extended internship, which should lead to a (tenured) faculty position but increasingly it’s treated as a long-term career path. RSE, in contrast, does not even have the pretence that it’s a stepping stone to a faculty job. A sustainable RSE position needs a career path, which means you need a mechanism for funding a pool of RSEs between grants (note: universities often have this for lab technicians).

                                                                                              The secondary problem is the salary. We (Microsoft Research Cambridge) pay starting RSEs (straight out of university) more than the UK academic salary scale pays experienced postdocs or lecturers[1]. RSEs generally expect to earn a salary that is comparable to a software engineer and that’s very hard in a university setting where the head of department will be paid less than an experienced software engineer. The last academic project I was on had a few software engineers being paid as part-time postdocs, so that they had time for consulting in the remaining time (a few others we got as contractors, but that was via DARPA money that is a bit more flexible).

                                                                                              The composition of these two is a killer. You need people who are paid more than most academics, who you are paying out of a central pool that’s covered by overhead. You can pay them much less than an industry salary but then you can’t hire experienced ones and you get a lot of turnover.

                                                                                              [1] Note for Americans: Lecturer in British academia is equivalent to somewhere between assistant and associate professor: tenured, but junior.

                                                                                              1. 2

                                                                                                Postdocs are all on fixed-term contracts.

                                                                                                Happy to talk more: what we’ve done is set up a Service Research Facility, which is basically a budget code that researchers can charge grant money against. So they “put a postdoc” on their grant application, then give us the money and get that many FTEs of our time. It also means that we can easily take on commercial consultancy, because you multiply the day rate by the full economic cost factor and charge that to the SRF. A downside is that we have to demonstrate that the SRF is committed to N*FTE salaries at the beginning of each budget year to get our salaries covered by the paymasters (in our case, the CS department), making it harder to be flexible about allocation and side work like software surgeries and teaching. On the plus side, it gives us a way to demonstrate the value of having RSEs while we work to put those longer-term streams in place.

                                                                                                The secondary problem is the salary […] so that they had time for consulting

                                                                                                You’re not wrong :). I started by topping mine up with external commercial consultancy (I’ve been in software engineering much longer than I’ve been in RSE), but managed to get up to a senior postdoc grade so that became unnecessary. I’m still on half what I’ve made elsewhere, of course, but it’s a livable salary.

                                                                                                Universities and adjacent institutions (Diamond Light Source, UKAEA, Met Office/ECMWF all pay more but not “competitive” more) aren’t going to soon be comparable to randomly-selected public companies or VC funded startups in terms of “the package”, and in fact I’d hate to think what changes would be made in the current political climate to achieve that goal. That means being an RSE has to have non-monetary incentives that being a FAANG doesn’t give: I’m here for the intellectual stimulation, not for the most dollars per semicolon.

                                                                                                A sustainable RSE position needs a career path, which means you need a mechanism for funding a pool of RSEs between grants (note: universities often have this for lab technicians).

                                                                                                I’m starting a DPhil (same meaning as PhD, different wording because Oxford) on exactly this topic in October: eliciting the value of RSEs and providing context for hiring, training, evaluating and progressing RSEs. I’ve found in conversations and panel discussions at venues like the RSE conference that some people have a “snobbish” attitude to the comparison with technicians, BTW. I’m not saying it’s accurate or fair, but they see making the software for research as a more academically-valid pursuit than running the machines for research.

                                                                                                1. 2

                                                                                                  Thanks, that’s very informative. Let me know if you’re in Cambridge (and pubs are allowed to open again) - I’ll introduce you to some of our SREs.

                                                                                              2. 2

                                                                                                Seeing as you seem to have experience in the field, from a very high level view, does the complaints about this project seem valid or not? I understand that one could only make an educated guess considering this is 15K lines, hotly debated, and also a developing situation (the politics… Whoo boy!), but I would love to have someone with experience calibrate the needle on the outrage-o-meter somewhat.

                                                                                                1. 1

                                                                                                  I haven’t examined the code, which is perhaps a lesson in itself.

                                                                                                  1. 1

                                                                                                    As a baseline I put the code through clang’s scan-build and it found 8 code flows where uninitialized variables may affect the model early in the run. It’s possible that not all them can realistically be triggered (it doesn’t know all dependencies between pieces of external data), but it’s not a great sign.

                                                                                                    Among others that’s a reasonable explanation why people report that even with well-defined random seeds they see different results, and I wouldn’t count “uninitialized variables” in the class of uniform randomness, so I’d be wary about just averaging it out.

                                                                                                2. 2

                                                                                                  If you cannot pay somebody much, give them a fancy title, e.g., “Research Software Engineering”. It’s purely an HR ploy.

                                                                                            2. 6

                                                                                              It’s you, the software engineering community, that is responsible for tools like C++ that look as if they were designed for shooting yourself in the foot.

                                                                                              There is very little impetus to build tools that are tolerant of non-expert programmers (golang being maybe the most famous semi-recent counterexample) without devolving entirely into simple toys (say, Scratch).

                                                                                              I actually agree with the author on this.

                                                                                              Let’s not even pretend that the only alternative to the absolutely mind-boggling engineering and design shit show that is C++ is “devolving entirely into simple toys”.

                                                                                              1. 1

                                                                                                Rust?

                                                                                                1. 1

                                                                                                  One option.

                                                                                              2. 4

                                                                                                I think you put it very well. Look: if there’s a hierarchy of importance I’m happy to put science far ahead of software development. But the fact remains: when it comes to producing scientific results using software, software developers do know a thing or two about how hard it is to fool yourself and we are rightly horrified at someone handwaving away lack of tests and input validation by “a non-programmer expert will look at this code and make sure not to hold it wrong”

                                                                                                I guess in that sense it’s not much different than the rampant misuse of statistics in science, it’s just that software misuse might be currently flying a little below the radar.

                                                                                                1. 4

                                                                                                  exactly. It is the job of the researcher to be aware of the limitation of his own limited ability to implement his model with a particular tool. To badly implement something then make grandiose claim that the result of said badly implemented model should inform decision that affect millions, is his own fault.

                                                                                                  You can’t blame a screw driver ‘community’ if you use it badly and poke yourself in the eye. Not even the lack of “do not poke eye with screwdriver” warning label counts as failure.

                                                                                                  1. 1

                                                                                                    This plays out in an interesting way at Google’s Research division. Whatever else you might think about the company, Google software engineers (SWEs) are generally pretty decent. Many of them are interested in ML research projects because they’re all the rage these days. The research teams, of course, just want to do research. But they can get professional SWEs to build their tools for them by letting them feel like they’re part of cutting edge research. So they end up with a mix of early-career SWEs building tools that aren’t inherently all that interesting or challenging but get used to do very interesting and impactful research work and a few more experienced SWEs who want to make the transition into doing research.

                                                                                                  1. 4

                                                                                                    For all the shitstorm, I see no actual bug report invalidating results in the open issues. Can anyone please point it out?

                                                                                                    Otherwise it feels all the testards and drive-by team leaders will teach researches better than to open up their code.

                                                                                                    1. 7

                                                                                                      This pretty much matches my impression as well – it’s hard to wonder if any of the people who wrote those “analyses” ever used – let alone wrote – simulation software.

                                                                                                      There’s enough incorrect material in them that writing a rebuttal would be tedious (plus I really ought to stop wasting time on Lobste.rs and get back to work…). But I just want to point out that the rhetoric matches the analysis.

                                                                                                      For example, in this article you see things like:

                                                                                                      “A few people have claimed I don’t understand models, as if Google has no experience with them.”

                                                                                                      unless the author of the article is Google, that’s pretty much irrelevant. Google has hundreds of thousands of employees, I bet quite a few of them don’t understand models. (The author of this article is definitely one of them, by the way).

                                                                                                      Edit: it’s nothing to be ashamed of, at any given moment there’s an infinite amount of things any of us doesn’t understand. But ranting about things one does understand usually gives better results.

                                                                                                      1. 4

                                                                                                        Are you saying that nondeterminism doesn’t matter because the model is supposed to be nondeterministic? Then why are they nevertheless fixing the nondeterminism bugs?

                                                                                                        Do you understand the value of reproducible research, which logically implies making the source code open in this case? Are you aware that Ferguson’s code wasn’t open source for over a decade, and that is part of the problem?

                                                                                                        1. 8

                                                                                                          To answer the nondeterminism part, normally you take a large set of runs and analyze them as a group.

                                                                                                          For example, a monte carlo of a gamma particle tunneling through radiation shielding is inherently non deterministic, however a large number of runs allows you to find the distance necessary for most if not all particles to be stopped safely. Nondeterminism is not an issue if the behaviors involved allow you to derive reproduceable results from the aggregate.

                                                                                                          That said, software bugs like incorrect branching can also be nondeterministic. The degree to how much they affected the simulation is often done through error propagation analysis or comparing the results before and after. Not all bugs are created equal - many can be obviously wrong but not “infect” the results enough to trash them. Still can muddy it tho.

                                                                                                          That’s why yes you fix bugs in nondeterministic models because the model is meant to be the only source of nondeterminism. Bugs have to be reduced out enough to avoid tainting the result set

                                                                                                          1. 3

                                                                                                            To answer the nondeterminism part, normally you take a large set of runs and analyze them as a group.

                                                                                                            If your simulation is meant to be nondeterministic, then good reproduceable science uses a strong PRNG and takes a seed from a configuration. You run it with a fixed set of seeds but can then reproduce the same results by providing the same set of seeds. If it’s not meant to be nondeterministic then it’s a bug and it’s impossible to know its severity without knowing more (but in C++, it can be any kind of undefined behaviour and so the end result can be complete nonsense).

                                                                                                            1. 2

                                                                                                              For example, a monte carlo of a gamma particle tunneling through radiation shielding is inherently non deterministic, however a large number of runs allows you to find the distance necessary for most if not all particles to be stopped safely.

                                                                                                              Sorry if I misunderstand, but surely being careful with when and who is calling your PRNG helps limit this, especially in a single-threaded case?

                                                                                                              Over in game development the issues around non-determinism are a pretty well-known if not always well-solved problem and have been for near two decades, at least.

                                                                                                              1. 8

                                                                                                                (Note: not parent).

                                                                                                                There are processes – I’m not sure if gamma particle tunneling is one of them because solid-state physics isn’t exactly my field, but if I recall things correctly, it is – which are inherently probabilistic. It’s supposed to give different results each time you run it, otherwise it’s not a very useful simulator, and I’m pretty sure I read at least one paper discussing various approaches to getting a useful source of randomness for this sort of software.

                                                                                                                (Edit: there are various ways to cope with this and reconcile the inherent determinism of a machine with the inherent probabilistic character of a physical process, assuming you really do have one that’s probabilistic. It’s not as simple as yeah, we just write simulators that give different results each time you write them.)

                                                                                                                In this particular (i.e. Ferguson’s code) case, the non-determinism (fancy name for a bug. It’s a bug) manifests itself as a constant-ish extra error term – you get curves that have the same shape but don’t coincide exactly, at least not over the duration where the model is likely to give useful results..

                                                                                                                Unfortunately, that’s exactly what you expect to get when doing stochastic process simulation, which is probably is a plausible reason why it wasn’t caught for a long time. This kind of error gets “folded” under the expected variation. That can have two outcomes:

                                                                                                                • If the errors are random, then averaging several runs will indeed cancel them out
                                                                                                                • If the errors are systematic, then averaging several runs will yield an extra (likely time-dependent) error factor, but it’s hard to say if that actually changes the simulation outcome significantly without doing an actual analysis.

                                                                                                                Thing is, the latter case is usually swept under the rug because these models are meant to investigate trends, not exact values. If you look at the two graphs ( https://github.com/mrc-ide/covid-sim/issues/116#issuecomment-617304550 – that’s actually the only substantial example of “non-determinancy” that the article cites), both of them say pretty much the same thing: there’s a period modest, then accelerated growth, that settles for a linear growth after 50-60 days.

                                                                                                                It’s not really relevant if you reach 200,000 deaths in 62 or in 68 days – not because “reproducible outcomes don’t matter” but because there is an inherent expectation that a model that’s supposed to tell you how a flu will spread over 90-150 days in a non-homogenous population of 40,000,000 people is not going to be accurate down to a few days.

                                                                                                                Edit: to clarify – I’m not saying that’s not a bug, it is. But it’s definitely not clear that its impact over the simulation results is enough to invalidate them – in fact, if I were to speculate (which is exactly what the authors of these critical articles do, since they don’t actually run any numbers, either) I’d say they probably don’t. The one bug report shows only two curves, and that’s not even enough to refute the authors’ argument that averaging enough runs will cancel out these errors.

                                                                                                                Edit: also to clarify – what parent comment is saying is, IMHO, completely correct. The only source of non-determinism in the result should be the non-determinism in the model, and bugs that introduce extra error factors should absolutely be fixed. However – and this is the erroneous message that these articles are sending – tainted result sets can still provide valid conclusions. In fact, many result sets from actual, physical measurements – let alone simulations – are tainted, and we still use them to make decisions every day.

                                                                                                            2. 3

                                                                                                              Do you understand the value of reproducible research, which logically implies making the source code open in this case? Are you aware that Ferguson’s code wasn’t open source for over a decade, and that is part of the problem?

                                                                                                              There is a culture problem in academia around this, but it is getting better and more journals are requiring source code with paper submissions.

                                                                                                              In this case, the model has been reproduced by researchers using different Probabilistic Programming Languages (Turing.jl and STAN), which is the bar it needed to reach. Discussion of the implementation quality isn’t really useful or scientifically interesting. It’s the inputs and modelling assumptions that are interesting.

                                                                                                              (Draft?) replication post here: https://turing.ml/dev/posts/2020-05-04-Imperial-Report13-analysis Code for that post is here: https://github.com/cambridge-mlg/Covid19

                                                                                                          2. 4

                                                                                                            There’s coverage from the first link in the submission.

                                                                                                            1. 0

                                                                                                              “Lockdown sceptics”, seriously? “Stay sceptical, but presuppose the conclusion you want to reach and find facts in support of it”?

                                                                                                              1. 6

                                                                                                                That’s neither here nor there, let’s stay on discussion about the issues they’ve found.

                                                                                                                1. -2

                                                                                                                  Yeah, they may have a perfectly good breakdown of issues in the simulation which affects results, I’m not discussing that. I didn’t take the time to read it (and probably won’t; the topic doesn’t interest me that much), and I should’ve been more clear that I’m not saying their findings are invalid. I just thought it was worth pointing out, and probably should be something people keep in mind while reading their review.

                                                                                                          1. 8

                                                                                                            I’m really not a big fan of RFCs in the workplace for most things, mostly because:

                                                                                                            • I prefer to have the discussion in the ticketing system or issue tracker–singe source of truth or many sources of lies and all that.
                                                                                                            • RFCs without accompanying implementation details can frequently be really loosy-goosey and not that useful.
                                                                                                            • Engineers will happily pad RFCs with other options that they aren’t seriously considering, giving fake rigor to them.
                                                                                                            • Management or senior engineers may just do an RFC and then say “okay, here’s what we’re doing instead” without actually following the recs of the RFC–or using it as just pro forma for what they were going to do anyways.
                                                                                                            • It can result in quality bikeshedding, especially if the document is open to a large peanut gallery.

                                                                                                            There’s another issue I have, somewhat brought up by the author:

                                                                                                            After I write the first draft I circulate it among a small group of peers I respect, my boss, etc. I request feedback at leisure and I check in every few days with a reminder. If no one responds after a while and there is little concern, I typically move forward with the proposed solution.

                                                                                                            “Every few days” implies a latency of decisionmaking on the order of a week or two. My experience has been either that people follow the RFC process and take way too long to converge to a solution, or that the people implementing it ignore the RFC. Both usually have knock-on effects.

                                                                                                            I think it’s a neat idea in theory, but I haven’t seen much beenfit from it on the teams where we’ve used it.

                                                                                                            1. 4

                                                                                                              All of these can definitely be issues. There’s still often value in going through the exercise of specifying solutions and listing alternatives/objections, and of capturing explicitly what decisions were made.

                                                                                                            1. 12

                                                                                                              I’ve ranted against JWT before, and that rant includes a few links and citations.

                                                                                                              Here’s a three-year-old discussion on this site against JWT.

                                                                                                              Here’s another anti-JWT article that goes into some detail.

                                                                                                              Here are a couple of articles explaining why JWTs as a “replacement” for session cookies are a bad idea.

                                                                                                              1. 2

                                                                                                                I would second the comment at the bottom of the email thread, would love your rant as a blog post.

                                                                                                                1. 1

                                                                                                                  How do you feel about solutions like Cognito?

                                                                                                                  1. 1

                                                                                                                    Mixed feelings. On the one hand, I generally push people to use someone else’s battle-tested auth system rather than rolling their own. On the other hand, I dislike ones that use JWTs since they inevitably mean using a JWT-parsing library with all the attendant risks.

                                                                                                                  2. 1

                                                                                                                    I mostly agree with this - I think for 99% of people, using JWTs is dangerous because it’s easy to miss one small thing and mess up the security of the implementation. And most people do it because it’s easy - when you’re writing a flask service nobody wants to handle setting up sessions because that involves a whole bunch of other components that aren’t built-in.

                                                                                                                    I’ve seen one implementation do it well: short-lived JWT access tokens, long lived session refresh tokens.

                                                                                                                    The access tokens are signed by an auth server and can be verified by microservices while the long lived refresh tokens are just session tokens - you can get an updated JWT by talking to the auth server again. This has the advantage that auth is centralized but any microservices only need to see the short lived token to verify the user’s identity. There are limitations (you can’t immediately expire all auth tokens - you can only expire the refresh tokens, so you may have living sessions until the latest auth token expires) but when done correctly it can be a useful technology.

                                                                                                                    1. 3

                                                                                                                      I’ll refer you back to my “rant” – this scheme is literally just one of (signed cookies | bearer tokens) but with extra work and less safety. JWT isn’t offering anything you wouldn’t get from one of those other options, except for the “feature” of people sometimes being able to literally Jedi-mind-trick their way into your services by waving their hand and saying “You don’t require me to have a valid signature” and your servers agreeing.

                                                                                                                      Or for a more comprehensive reply, read any of the “don’t use JWT for session tokens” articles. They tend to cover this scheme and debunk its usefulness.

                                                                                                                      1. 2

                                                                                                                        I think for 99% of people, using JWTs is dangerous because it’s easy to miss one small thing and mess up the security of the implementation.

                                                                                                                        I think it’s also worth contrasting between consuming JWTs and creating them. Lots of identity providers create them and those JWTs should be pretty bullet proof (no none algos, supporting good encryption protocols). It’s their job, after all.

                                                                                                                        Rolling your own JWT seems far more problematic to me.

                                                                                                                        There are limitations (you can’t immediately expire all auth tokens - you can only expire the refresh tokens, so you may have living sessions until the latest auth token expires) but when done correctly it can be a useful technology.

                                                                                                                        This post (full disclosure, written the CEO of my company) talks about JWT revocation options. May be of interest to you: https://fusionauth.io/learn/expert-advice/tokens/revoking-jwts

                                                                                                                    1. 6

                                                                                                                      I’ve been doing interviews for a large Nordic Bank the last couple of years. Many of the developers applying there are a bit older. That’s not negative in itself. It usually means more experience. But, one major concern for that large bank, when applicants are older, is that the developer is merely searching out this “stable” job to kind of breeze through, without too much effort, to retirement.

                                                                                                                      So, my two cents; if you are an older developer (like myself) try to be clear about why you are searching out that particular job.

                                                                                                                      Ps. By the way. In the Nordics we have strong unions, and labour laws, large companies can’t really fire people for not being productive enough. So this piece of advice might not be applicable in other places.

                                                                                                                      1. 5

                                                                                                                        It’s absolutely applicable in the United States.

                                                                                                                        The other piece is a concern that older people can’t be trained in the company’s local customs/culture, which can lead to massive communication and productivity problems even with the best of intentions.

                                                                                                                        1. 3

                                                                                                                          older people can’t be trained in the company’s local customs/culture

                                                                                                                          I’m sure you didn’t mean it this way, but to me, this sounds like when companies talk of ‘culture’ as an excuse for working stupidly long hours without a care for your life outside work, obligatory drinking and partying, and generally trying to pretend that a toxic workplace environment centred around a specific demographic is a positive thing. I can’t imagine what other sort of workplace culture an older person would have difficulty learning/fitting into.

                                                                                                                          I’d love it if you could expand a little on what you really mean.

                                                                                                                          1. 5

                                                                                                                            Every company builds its own culture. And yes, that matters.

                                                                                                                            • Some have strict chain of command. You do not go over your manager’s head to the top levels. Ever.
                                                                                                                            • Some have open access. Call the CEO whenever you have something you think he should hear.
                                                                                                                            • Some require all problems be presented with a possible, even if terrible, solution, at least as a starting point.
                                                                                                                            • Some companies have deep sharing cultures. If you can do something, you better be teaching other people how to do your job.
                                                                                                                            • Some allow people to attain mastery of their own space because it’s more efficient that way.
                                                                                                                            • Some companies actually do respect feedback.
                                                                                                                            • Some companies require people to only bring issues they know for certain are actually issues, to avoid distractions or rabbit-hole fixes.

                                                                                                                            An employee who has spent years working with that last bullet point is going to find it hard to work in a place where all issues are brought forward for at least brief discussion. That’s a cultural mismatch, and it’s not anything except an inability to utilize a person as trained.

                                                                                                                            1. 0

                                                                                                                              An employee who has spent years working with that last bullet point is going to find it hard to work in a place where all issues are brought forward for at least brief discussion.

                                                                                                                              People can just learn to do thing differently. Do people find it hard to work in different ways at different places? If not, why is this a reason to not hire somebody. Putting such a strong requirement on something that can be changed anyway seems like a good way to have to go through more people than you have to during recruitment.

                                                                                                                              1. 3

                                                                                                                                Do people find it hard to work in different ways at different places?

                                                                                                                                Yes.

                                                                                                                          2. 2

                                                                                                                            I work at a place with a strong union in the US.

                                                                                                                            1. 5

                                                                                                                              that seems to be increasingly rare, for programming jobs anyways. or are you doing something completely different than that?

                                                                                                                              1. 4

                                                                                                                                What has that experience been like?

                                                                                                                                1. 1

                                                                                                                                  I do, too. Worker protection is rare in US, though. Their comment is still mostly true.

                                                                                                                                2. 1

                                                                                                                                  older people can’t be trained in the company’s local customs/culture

                                                                                                                                  I feel like some times when a company say someone isn’t a “culture fit”, they’re just using coded language to discriminate against a worker. Age discrimination is illegal in the US, but not hiring someone because of “culture fit” is totally fine. Similar with race, sex, etc.

                                                                                                                                  1. 3

                                                                                                                                    See my answer here: https://lobste.rs/s/qu40ze/how_prepare_for_losing_your_programming#c_omhxsj

                                                                                                                                    Companies know they have their own cultures. A person’s past experiences or levels of aggression (or lack thereof) may be incompatible with the way work is done at a company. It has nothing to do with being old, just with being trained.

                                                                                                                                3. 2

                                                                                                                                  Then ask yourself, is your IT landscape one with recent commodity frameworks and infrastructure, or is there a lot of in-house software and infrastructure that might take someone half a year, maybe longer to understand and grow into?

                                                                                                                                  Is your employer the kind of place that needs (and retains) ambitious people who shake things up, or do you need people who are willing to maintain legacy software without complaining too much?

                                                                                                                                  If you’re willing to invoke the stereotype of the greybeard who wants to coast until retirement, then you should be fair and invoke the stereotype of the ambitious fresh graduate who rewrites everything in Ruby on Rails, then leaves after three years, leaving the organization with the pieces.

                                                                                                                                  Edit: I should’ve realized that you point out this stereotype to warn people of stereotypes that work against them, not that you’ve internalized this stereotype yourself