1. 13

    Sometimes, Programs That Use Transactions… Are Worse

    You ever see a reference that is such a deep cut you feel like it’s written for you, specifically?

    1. 9

      I am personally delighted every time someone catches these. :)

      1. 2

        Haha, amazing! Thank you for introducing me to this.

      1. 4

        I liked the analysis, but the part about ACID and Snapshot Isolation reads a bit off:

        Snapshot isolation is a reasonably strong consistency model, but claiming that snapshot isolation is “full ACID” is questionable.

        Marketing usage might be deceptive, but to me this reads like you’re saying that only Serializable transactions are ACID. But even Serializable transactions exhibits real-time anomalies. Then you’re left with strict serializability, which afaik only Spanner guarantees for distributed settings. Can Postgres still claim to have ACID transactions? Is Spanner then the only player with “full ACID” transactions?

        Although the “I” in ACID means Isolation, there are degrees. Both Read Committed and Strict Serializable transactions are, to me, “fully ACID”.

        1. 12

          I’d disagree! ACID isn’t… really a well-defined property, which is why I couch it carefully in this report, but in general ACID “isolation” is understood to have something to do with transactional interleaving and equivalence to a serial history. Realtime-only anomalies in serializability don’t violate isolation, because they don’t create (visible) interleavings of operations across transactions. In this sense, serializability is the weakest of several consistency models which provides ACID “I”.

          Consistency is a bit of a different beast, because there are application-level invariants which could be violated by serializability, but preserved under, say, strong session or strict-1SR. For those cases, yeah, you could argue serializable isn’t ACID either. That’s part of why I don’t like “ACID” as a descriptor, but people keep using it, so… here we are.

          1. 2

            Fair enough! It doesn’t help that the literature can call Snapshot Isolation (or any other) both an “Isolation level” and a “Consistency criterion”, although I see the first one less and less nowadays.

            That’s part of why I don’t like “ACID” as a descriptor

            It’s true that ACID transactions can mean different things to different people, which I guess that’s why it’s the perfect marketing material. With that… it’s on us to come up with a new hip term to sell to clients!

        1. 14

          Another (over my head) but nice, thoughtful job!

          @aphyr, knowing that you spend a non-trivial amount of time doing your analysis, what prompted the un-compensated analysis?

          1. 46

            A few things came together! One is that people are always mentioning Jepsen and MongoDB together, and asking what it does now, and I keep fumbling the ball. Another is that the MongoDB Jepsen test suite is one that people frequently try to run, and when it fails, they file GitHub issues asking what’s wrong, so it’s been in the back of my head like “yeah, I need to go dig into that at some point.”

            About… maybe a month ago, Evan Weaver, from FaunaDB, sent me a link to the MongoDB Jepsen page which accidentally forgot to talk about default behavior. I was busy and forgot about it until Jepsen got tagged in a Twitter thread where a MongoDB developer advocate said “We are passing the Jepsen test suite and it was back in 2017 already. So, no, MongoDB is not losing anything if you know what you are doing.”, and linked to the page again! THAT was like oh, yeah, I REALLY gotta do this.


            I’d just finished a full rewrite of Jepsen’s generator system and I needed a project to use as a proving ground, to make sure it was actually usable before release. Dug into the MongoDB test suite code and realized I couldn’t get it to run either–it’d accrued a bunch of code for other environments, and I think somewhere along the way it stopped working with a standard Jepsen environment. I started a rewrite expecting I’d basically just confirm transactions were SI, but then… I found the defaults weren’t, and then even when I fixed the test suite to use the correct safety levels THOSE looked broken, and it just kinda snowballed from there.

          1. 2

            The date is today’s date, but 2019, is this a year old?

            1. 10

              This is what I get for writing “2019-TODO” in October and only updating the TODO part before release!

              Should be fixed momentarily, I’ve just been waiting on gcloud to pick up the changes. Takes forever.

              1. 1

                A glimpse into how long it takes to work on these! Very curious about this now, actually…

                1. 35

                  Each report generally involves 2-3 months of my full-time work before the report is “finished”, and from that point, there’s an up-to-three-month delay before publication, which gives folks a chance to make bugfixes, inform users of risks, test and cut releases, etc. I check in a day or two prior to publication to make last-minute updates–“this issue is resolved by version foo”, etc.

                  There’s generally 1-2 days of research and documentation review, about a week to write a basic “it turns on” test harness for a new database, and another 2-16 weeks of exploration, refinement, new workloads, failure modes, etc. Throughout this process I’m in close contact with the client, showing them my findings, asking for help understanding expected behavior, suggesting documentation fixes or possible algorithmic issues, debating whether something is “really broken”, etc.

                  I spend about a week actually writing each report–sometimes multiple weeks, depending on the complexity. Sometimes I write at the end based on my “lab notebook”–really, a text logfile. Other times, I try to write throughout the process. Concurrent with the writeup, there’s a lot of work involved in collecting type specimens for each error, condensing those specimens to something you can actually understand in the paper, writing up bug tracker tickets, figuring out which of the 20-odd development builds I tested included/fixed which bugs, etc.

                  Client comments can add additional weeks of back-and-forth; with some clients I just get a “looks good, ship it!”, and with others, we go through literally dozens of drafts fine-tuning language and debating how to visualize or label different issues. I review my own work for structure, language flow, and typos with at least three passes, spoken out loud. For a ~10 page paper, that’s multiple hours, possibly more than a day, of work each pass.

                  And despite my own proofreading, as well as having a half-dozen client reviewers, and occasionally peers, we still sometimes miss obvious things like “what year is it”! ;-)

                  1. 1

                    I’ve always wondered about your process to get these out. What percentage of the systems you test are open as opposed to proprietary systems about which you don’t post online?

                    1. 1

                      Good question! I was worried, when I started out, that folks would try to weasel their way into keeping analyses from the public eye, and wrote a policy specifically to address this: https://jepsen.io/ethics.

                      Whether a system is open vs proprietary is a different question from whether I release a public analysis. I test completely open-source systems and completely closed-source ones alike, and they both get the same treatment at release time: test suite, examples, and the report are free for everyone.

                      Basically, when folks ask for Jepsen work, they choose whether they’d like an actual public analysis or not. If they say yes, it is (I mean, assuming I can physically do the work, get paid, etc) getting published. Almost everyone chooses public. I think the last private Jepsen-analysis work I did was… one week in 2017?

                      In addition, sometimes I do internal consulting, like reading docs or talking to engineers about their plans for a DB or infra. And there’s classes, trainings, and internal tech talks, most of which I don’t talk about publicly. I don’t think that stuff matters as much as my analysis work though–it’s generally not of public interest that I taught at FooCorp. :)

                      1. 1

                        Thanks aphyr! Nice to know that the percentage is quite heavy on the open side.

                        Since you mentioned trainings, would love to be in one of our distsys classes. I’m not sure if my employer would be willing to sponsor a class but if I could attend one of your classes that you organize in the open or with employers, that’d be great! :)

                        1. 1

                          They’re welcome to talk to me about running a class–I don’t have any open sessions planned right now though.

            1. 16

              Am I misunderstanding, or is this quote:

              unless you’re a doctor, or a neuroscience researcher, or a bigot (but I repeat myself).

              saying doctors and neuroscience researchers are bigots? Not commenting on the topic, but just double checking my reading comprehension.

              1. 7

                Trans people want to be able to determine their gender themselves, which contradicts the medical view that it is biologically given. Of course trans people hate doctors, and doctors are confused with how to deal with trans people.

                I consider the medical knowledge quite accurate in describing the things as they are, and i consider the view of transgender people as accurate in describing how they feel.

                1. 13

                  I think this is a little bit too simplistic. The problem is really that trans people (in my experience) tend to be afraid that doctors will deny necessary medical treatment (hormone therapy, mostly) and the letters that many governments require to “prove” that someone is “really trans”, and that doctors tend to be dismissive of women and trans people regarding totally unrelated medical problems - especially pain. (Off the to of my head: The Atlantic reported on this.)

                  I consider the medical knowledge quite accurate in describing the things as they are, and i consider the view of transgender people as accurate in describing how they feel.

                  Whoof. I’d appreciate a clarification on this because I really want to engage with this in a useful way, but it sure does read as “Trans people think they’re X but doctors (and I) know they’re really Y.”

                  1. 6

                    [Update: I agree that dealing with the medical system is a PITA]

                    What else is body dysphoria if not the discrepancy of what you observe yourself to be and what you feel you are?

                    I intentionally mirrored the phrasing of both parts to show that they are equally valid, even if they are incompatible to some extent. Science is the tool, our well-being the means.

                    1. 1

                      Ah, that makes sense, I see. A deft prosaic move I entirely missed. Clearly I need some coffee :)

                    2. 2

                      Alternatively, why is it suddenly wrong for health care professionals to really evaluate people carefully before prescribing drugs? I really feel like we overprescribe things like anti-depressants and ADHD medication and that should really concern people.

                      What is so wrong with simply offering the option of therapy to help someone be okay with their biological gender? They can also offer the option to transition, but both should be equal options. I realize some may equivocate that with “pray the gay away” type therapy for those not wishing to be homosexuals, but what if it’s not?

                      Gender dysphoria is treated in radically different ways to other types of dysphoria. We don’t suggest people with Body Integrity Disorder amputate their arms and limbs as the first means of treatment. We don’t accept people with anorexia as being healthy in their view of the world/themselves. Both of these types of dysphoria deals with the metaphysical dissonance from one’s self reflective empirical view of the world to an outside classification based view of the world. Yet for gender dysphoria, why is it okay to offer treatment that helps an individual conform to the outwards portrayal of a given empirical classification?

                      1. 11

                        Actually, people suffering from ADHD are notoriously under prescribed due to this myth. I went 25 years without medication due to the ADHD medication panic. Do you know how many opportunities passed me by in those 25 years? These “careful evaluations” are mass hysteria nanny state garbage. Doctors and patients do not need an arbitrary set of hurdles set by an uninformed public for with diagnosing and treating individuals. Imagine if you needed to go through physical therapy before getting a cast for your broken arm. The doctor says, all modern research shows that it would actually be better for you to get the cast before physical therapy but some lawmaker decided that was illegal, so you’ll have to go to therapy. Good luck getting someone with ADHD to follow treatment instructions without medication.

                        1. 4

                          Anorexia and “gender dysphoria” are nothing alike.

                          Try living close to two people who have or have had these conditions and you’d appreciate that.

                          Both people have received treatment according to modern medical and scientific consensus, and luckily not from the op-ed pages of Quillette.

                          1. 3

                            I mean, that’s anecdotal. I have actually had close friends in both of those camps and I think there are similarities, but my experience is also anecdotal (and by definition empirical).

                            Here’s an opinion from a woman who thinks there’s similarities who has anorexia:


                            I think her views are worth reading, but that’s not hard data. What this post is about is data science. To get hard data, you have to make classifications, and look at what you pull back, and see if there is a close enough R value to have a correlation, and even then you might not have causation. That’s a whole different level of introspection.

                            The trouble is it’s getting increasingly difficult to even do the research. If we can only identify someone by their self-identifcation of gender and not their biological sex because that’s bigoted, we don’t even get accurate data to show, “For thing x, there is no difference between a trans woman and a cis woman” or “For thing y, there is a huge difference between a trans woman and a cis woman.”

                            When people get so angry and just keep throwing up anecdotes and restrictions instead of larger data sets and controls, and research, we’re not going to be able to get real meaningful data. And as scientists and engineers, we should value that hard data, no matter what uncomfortable truths it might reveal.

                            1. 5

                              You’re comparing the current body of suggested treatment against anecdotal evidence, don’t you wonder if that’s not at least a little indicative of the frailty of your argument? For the record, anorexia kills when they embody their desired presentation, being trans does not. Anorexia inherently is incapable of being resolved through allowing them to present themselves the way they desire, because doing so kills them. You can’t meaningfully compare that to someone who can live a happy life after treatment. Furthermore, therapy isn’t going to do diddly for a lot of these issues in the same way that you probably would have a great deal of trouble convincing yourself tomorrow that you’re the opposite gender you are. If it’s surprisingly easy for you, congratulations you may be genderfluid, and that’s cool, but most people aren’t. Most people go through progressive rounds of trying things, seeing if that level of treatment works for them. They try HRT, which works for many. Then they try non-surgical cosmetic modifications. etc, etc.

                              1. 5

                                The Federalist has its own agenda when it comes to transgender issues: https://thefederalist.com/?s=transgender

                                The mystical doctrines of transgenderism exemplify modern self-worship, in which the human replaces the divine dictates of revealed religion as the source of meaning. source

                                It’s not a publication that’s a friend of science.

                                The trouble is it’s getting increasingly difficult to even do the research.

                                Citation most definitely needed. How can a (relatively) broader acceptance in Western society of LGBTQ+ people and issues cause less research to be performed?

                      2. 4

                        I stumbled over the same sentence, and had to re-read it a few times. I thought it was unnecessary and detracted from the piece, tbh

                        1. 8

                          A lot of trans people experience a lot of ignorance and discrimination from medical personnel, especially doctors. In my experience this tends to stem from doctors thinking that they know more about people’s own gender and self-image than those people, and using their position as gatekeepers of things like legal gender changes and hormones to force trans people to conform to their often narrow definitions of transgender.

                          Obviously not all doctors are bigots, but they do have a… reputation.

                          1. 10

                            not all doctors are bigots, but they do have a… reputation.

                            Of all people, I would think that trans women would be the least likely to engage in this kind of callous generalization. Why do you do that?

                            1. 9

                              Safety. The less I interact with new doctors, the less likely I am to have bad things happen to me regarding my prescriptions, etc. Same thing with therapists, and not just for trans people; lots of people, though mostly women, have the experience of having to search for a doctor or therapist who will take them seriously. It’s actually a pretty well documented phenomenon, and there have been several studies showing that marginalized people (women, trans people, people of color, and of course people at the intersections of those identities) are taken less seriously by medicine.

                              1. -1

                                Oh no, it makes sense. The biological model of gender is supported by massive amount of physical evidence which the doctors were educated in and see in their patients everyday. The new ideas of gender identity originated in far-left colleges with basically no hard evidence backing them: just psychological studies (“soft science”) of what trans people believe about themselves and tell them. There’s some neurological studies showing mixed results. Even they suggest their brain works differently while the rest of them is the biological gender.

                                So, it makes sense that most doctors trained on and seeing confirmation of the biological model of gender would reject trans identity claims in favor of biological model. That’s the scientific thing to do until there’s more evidence of the new claims than the old claims. Unlike at some universities, the P.C. folks can’t just shout down and eject the doctors. Not yet anyway.

                                1. 24

                                  The new ideas of gender identity originated in far-left colleges

                                  New compared to what? I mean, this stuff was considered radical and new in the 1920s (it’s literally what the Nazis used for their book-burnings).

                                  As to mechanism - it is definitely not solved and I’m not even aware of any good evidence-based theories.

                                  It is, however very well-established:

                                  • That gender dysphoria (the distressing belief that your body is Wrong) is real
                                  • Strongly correlated with suicide (41% of sufferers attempt it vs 1.6% of general pop - yeah, the study is from a gender studies school, but who else is going to look into it?)

                                  It’s less strongly established (but there’s good evidence for):

                                  • Suicide risk is markedly reduced among those who can ‘pass’
                                  • Surgery makes it much, much easier to pass
                                  • Obtaining surgery (or any other treatment) requires you to fit a set of diagnostic criteria. Evaluating these criteria takes 3+ years in many jurisdictions, with no guarantee of success.

                                  Many - many - people with a plausible belief that treatment would help them get denied because they don’t meet the precise diagnostic criteria. Getting that news after 3 years of fighting for it is a pretty serious blow.

                                  As a result, they are pretty angry, on the whole, at doctors who do not bend over backwards to fit them into the diagnostic criteria, because they believe (correctly, IMO) that the criteria are both too narrow and require excessive investigation (one year being seen as more reasonable than three).

                                  1. 8

                                    I want to add that queer activism is probably compensating/self-dealing behavior for people with gender dysphoria, and is probably not representative of transgender people in general.

                                    Anecdotal: Asperger/Autism, depersonalisation and self-harm also seem quite common for trans people.

                                    1. 9

                                      Can I ask what you mean by “queer activism?” I mean… yeah, people who have gender dysphoria often want to alleviate that dysphoria, and in our current society that requires being an activist, at least in the small (advocating for yourself with your doctor, not going to stores/working at companies with transmisic policies, etc.)

                                      1. 5

                                        Some people are really obsessed with queer and identity politics, so much that their entire existence revolves around being trans, non-binary or ally. I call them activists.

                                        And then there are people that just used to have a different gender at birth and are now living a life as the gender they wished for.

                                        There is a clear line - the second category does not like to be called ‘trans’, they are ‘girls’ or ‘boys’. The non-activist queer person is the one who is just neither a ‘girl’ girl nor a ‘boy’ boy. They defeat the gender binary without defining themselves as queer.

                                        1. 8

                                          I see what you mean, but I think it’s a little more complicated than that, for a lot of people.

                                          I’m a woman, definitively so, and I actually pass as cis most of the time, but I do like people to know that I’m trans sometimes, especially people like nickpsecurity, because it upends their idea of what a trans person is. I also have experiences that some women, mostly cis women but also trans women who were able to block their first puberty and start hormones at the socially appropriate time, don’t have, and identifying as “trans” is a nice way to get into groups with people who share and can understand my experiences.

                                          In any case, I’m realizing that this whole thread is pretty far off topic. Thanks for humouring me and clarifying your posts!

                                          1. 5

                                            I’ve been reading everything in what little time I had. I’m mostly holding off on replying to this thread until I have some sleep or not at all until I contemplate the highly-insightful replies I got. I should mention…

                                            “but I do like people to know that I’m trans sometimes, especially people like nickpsecurity, because it upends their idea of what a trans person is”

                                            …that I figured you were a trans woman after a few comments. @cadey, too. There’s a difference between how you all speak and write versus most of the thousands of biological women I’ve known. I’m not saying it’s in any way highly accurate. My intuition has almost exclusively been exposed to trans women who are tech geeks on Lobsters: the only place I run into trans people on the regular (that I know of). There’s definitely a difference that some part of my mind saw which captivated me further to read each of your posts. That’s on top of the fact that you were smart, thoughtful people whose comments demanded my attention regardless of source.

                                            That said, the quote is still correct in that I’m learning about you all. How I learn takes exposure and real conversations with people to get through all the biases and cultural BS. The South, maybe the world in general, is too hateful on trans people for me to do that. They react with instant withdrawing or combat likely assuming I’m like the worse folks they encounter. I do appreciate the replies from trans folks on Lobsters who have very much broadened my understanding of things. Also, quite a few have set a hell of an example for how to do this vs others I’ve met. @Irene especially comes to mind as she was one half of what set that in motion in my head.

                                            All I’ll say for now. Too sleepy from work to do much more than this off top of my sleepy head comment. Hope yall have a good night. :)

                                            1. 5

                                              How I learn takes exposure and real conversations with people to get through all the biases and cultural BS. The South, maybe the world in general, is too hateful on trans people for me to do that. They react with instant withdrawing or combat likely assuming I’m like the worse folks they encounter. I do appreciate the replies from trans folks on Lobsters who have very much broadened my understanding of things. Also, quite a few have set a hell of an example for how to do this vs others I’ve met. @Irene especially comes to mind as she was one half of what set that in motion in my head.

                                              Thank you for at least trying to learn and be open minded. As a fellow Southerner (I’m a Tarheel!), I definitely agree that there are some real cultural differences. Being trans pushed me towards overcoming a lot of those, but all we can ever do is be willing to learn.

                                              I’m sorry I judged you too quickly, and I hope we can learn more from each other in the future. I’m also always happy to talk privately about this stuff; especially keeping in mind your comments about cancel culture, it can be easier to learn without an audience.

                                      2. 6

                                        I see parallels between modern queer activism and Zionism - in that I can understand what motivates it (their lives are plausibly threatened), and I can’t blame its participants for the conclusions they’ve come to (I’ve not had to find out how far I’d go to protect myself), but I’m still deeply upset by what they are doing.

                                        1. 16

                                          I’m still deeply upset by what they are doing.

                                          Like @NoraCodes said: What is the issue here? My queer activism is centered in things like:

                                          • I should be able to see my partner in the hospital
                                          • Everyone should have access to HIV prevention
                                          • People should be called by the names and pronouns they like
                                          • You should be able to get hormones, hair treatment, and surgery if it helps you live a fulfilling life
                                          • I’d like folks to stop yelling “faggot” at me on the street, and I want to buy groceries without people threatening to kill me for being gay.
                                          • It’d be cool if kids didn’t grow up feeling like they were fundamentally unlovable and worthless because they were queer
                                          • Let’s… not murder people for their sexuality or gender, ok?
                                          • You should be able to use the restroom without being assaulted because you didn’t seem masc or femme enough to be there

                                          I hope these aren’t too upsetting for you.

                                          1. 2

                                            Sorry, that was hastily written and needlessly inflammatory.

                                            I’m referring specifically to cancel culture, especially the outrage-porn-driven variety, and especially especially where it results in online vigilantism.

                                            One poorly-thought-out tweet gets screenshotted and cross-posted to a bunch of other networks. Within a week, the author is either publicly begging for forgiveness, or unemployed. They can’t use notifications on their phone anymore due to the volume of spam.

                                            Even in the case where this person genuinely held an offensive view, I don’t think that’s a good outcome.

                                            The people who are doing this hold a justified belief that lives are at risk, and hold that any amount of collateral damage is acceptable defending themselves.

                                            1. 3

                                              A particularly egregious example of this is getting speakers no-platformed from university campuses.

                                              Universities are perhaps the only place where I really couldn’t support any form of no-platforming.

                                              They are - literally - a place set aside for sorting bad ideas from good ones.

                                              1. 2

                                                Understood; “cancel culture” is a totally different topic and truly completely irrelevant here. Nobody is being cancelled and nobody has threatened to do that. It is not unique to nor synonymous with queer people, activism, or even progressives.

                                                1. 2

                                                  I’d argue it’s a subtype of ‘activism’ (though agree it’s not limited to progressives or queer peeps).

                                                  Fair point that it’s a method largely orthogonal to the cause. However, it’s harder to argue that it’s not a popular method in queer activism (popular in part, IMO, because the stakes are high enough that the collateral damage is judged acceptable).

                                                  1. 2

                                                    Understood; “cancel culture” is a totally different topic and truly completely irrelevant here.

                                                    I’ll note there was an entire meta dedicated to me after one of my run-ins with P.C. culture here. One person, with some upvotes, wanted my entire user tree banned. Seeing mob-like behavior, I stopped inviting folks to avoid reputational or other collateral damage for them. The metas always have people voting in large numbers in that direction or similar paths. We mostly downvote or filter, though, since our moderation strategy does it better than what much of that crowd wants.

                                                    So, de-platforming is not irrelevant when there’s people actively trying to make it happen. It’s always worth remembering they’re here like they are in many places, esp colleges. I don’t worry about it since I know our admin and mods. I just keep an eye on it while representing the dissenting side who are not present or too worried to speak in as civil way as I can.

                                              2. 11

                                                Why? Like, truly, what about me taking estradiol and changing my name is such a huge problem for you?

                                                1. 2

                                                  At least by my book, that’s living, not activism (see also my response to /u/aphyr).

                                                  1. 4

                                                    I say this gently, but this is the most cis+heterosexual thing I have read all day.

                                                    1. 1

                                                      Fair point to the ‘living is activism’ crowd.

                                                2. 4

                                                  What do you mean by the term “Zionism”? The original quasi-nationalist ideals of Theodor Herzl, or the modern semi-articulated ideology of the modern Israeli right?

                                                  1. 3

                                                    Specifically, I mean the combination of two ideas popular around the founding time of Israel (and key to the US fundraising which enabled their defense immediately post-foundation):

                                                    • That a homeland was the only way to safeguard their lives, and
                                                    • That any action is morally defensible when protecting your own life.
                                                    1. 1

                                                      Are there trans activists advocating (implied) violence in this way?

                                                      1. 1

                                                        I know several. Thankfully, none of them are any good at violence (presumably the main reason they stick to advocating it).

                                                        1. 1

                                                          Wow. That’s not part of the LGBTQ+ rhetoric here in Sweden at all.

                                              3. 1

                                                “New compared to what?”

                                                That is unclear looking back on it. I’m not talking about existence of trans people, gender dysphoria (well established), etc. Even Ben Shapiro of all people often cites gender dysphoria (“mental illness” in his words) and suicide risk in his arguments against accepting trans identity. They’re just facts that can be used to argue for or against many things.

                                                Let me try again. I’m talking about the new-ish rule that sex and gender are different. It appears that somewhere from half to a majority of the U.S. believe that your sex is your gender. This was prevalent belief for most of human history. Noting the biological differences was entire reason the gender words/pronouns were invented. These meanings got into lots of systems and institutions people later built.

                                                Then, the newer thing came along starting in universities that I can tell. Their studies and internal debates led them to redefine gender, male, female, etc to mean entirely different things. Those views started spreading to a lot of other places. Then those with those views wanted everyone, esp general public outside these universities and groups, to similarly redefine the existing words to match the newer meaning. Then, many are surprised and/or outraged that those people continued using the words for the popular beliefs they were actually designed for. They’ll have to convince those people the new meanings are true with evidence that outweighs all their experiences supporting that the biological definition is true. If that can happen, it requires different approach than whatever they did to preach to the choir.

                                                The other thing I noticed was an inconsistency. Certain words or symbols, esp that were associated with hate, are forever banned from being reused in new situations. The people that advocated that explained to me that the words had a lot of baggage, established meaning, widespread belief, history, etc that came with them. We can’t separate that from the word anymore. So, we have to come up with new words. Then, some of those same people were telling me that we’re redefining gender to mean something entirely different than what hundreds of millions of men and women think it means and how they’ve used it for over a thousand years. Seems inconsistent, even hopeless. So, I advocated new labels, pronouns, or just modifiers to indicate the difference to increase chance of adoption.

                                                “ because they don’t meet the precise diagnostic criteria”

                                                I haven’t researched much about the surgeries or their rules. I leave others to decide that stuff since I don’t have an informed opinion on it.

                                                1. 23

                                                  Then, the newer thing came along starting in universities that I can tell. Their studies and internal debates led them to redefine gender, male, female, etc to mean entirely different things.

                                                  This series of posts is a good example of why having classes on gender and sexuality can be helpful. Sex and gender have a complex history, and studying a bit of that history can give you an appreciation for your current cultural perspective. For instance, Navajo has at least four (perhaps five) genders, and pre-colonial gender systems throughout the Americas included cultural scripts & roles for people falling outside what you and I might describe as the male/female binary. Some associate healer or shamanic roles with third-gender persons. Some have specific ceremonies for gender transition. Pre-colonial Hawaiians revered gender-non-conforming people as sacred educators. India has a well-known role called “hijra”, for male or intersex people who dress in feminine clothing, as well as “sādhin”, who wear men’s clothing and short hair.

                                                  Indeed, modern queer roles in the US (e.g. “a gay man”, “a transgender woman”, “a non-binary person”), just like normative roles (e.g. “a straight woman”) need to be understood not only as the product of innate identity, but also as a consequence of our culture, which inherited a rigid gender and sexual binary in part from Victorian England. Modern queer roles in the US are controlled by and also subvert that gender system. They’re a product of individual and collective experimentation, friendships, families, bars, media, and yes, liberal (and conservative!) universities, operating in the context of a mainly-heterosexual, mainly binary-gender culture. This is also nothing new: there’s thousands of years of precedent outside the gender and sexual system you’re thinking of as universal and eternal.

                                                  There’s a lot of history I’m eliding here, and this comment is already too long–but hopefully this inspires you to read and learn a bit more!

                                                  TL;DR: sexuality has never been simple.

                                                  1. 2

                                                    Preach 🙏

                                                  2. 11

                                                    Let me try again. I’m talking about the new-ish rule that sex and gender are different.

                                                    Again, this is “new-ish” in the sense that it’s a product of anthropologists around the 1920s finding that gender in some societies (that were either newly contacted at that time, or being newly re-studied) didn’t work exactly the same as it did in European society. They wanted to keep sex as an etic category (seen from the outside) while using gender as the emic category (as seen from inside the society). So it’s new if your sense of novelty has a 100 year lag; though I suppose the distinction didn’t make it out of anthropology departments and into the rest of the university until maybe the 60s?

                                                    1. 3

                                                      Imagine if non-programmers talked about 100 year old discoveries in computer science as “new-ish”.

                                                    2. 8

                                                      This was prevalent belief for most of human history

                                                      It appears that somewhere from half to a majority of the U.S. believe that your sex is your gender

                                                      Historically, it’s more-or-less always been true that there’s a minority ‘queer fringe’, who live with varying degrees of acceptance alongside a society which doesn’t really understand them. There’s ~always been a majority belief that your sex is also your social role.

                                                      The major conflict in the west (unfolding in public for at least 60+ years) is over access (by that minority) to mainstream society - in particular, access to jobs, housing and healthcare. EG Being openly gay used to get you fired and/or evicted, and the government deliberately suppressed info on AIDS treatment because it was a ‘gay person’ disease.

                                                      The mainstream has accepted some of the larger segments of the queer community. However, the smaller populations (inc trans people) weren’t able to really fight effectively and didn’t win many rights/recognition at that time.

                                                      Then, the newer thing came along starting in universities that I can tell

                                                      That’s where it started getting pushed out into public life. I think it’s pretty unlikely that nobody had come up with this idea before - it’s immediately obvious to (for instance) a gay man that they do not fit the gender role assigned to their sex, because chasing women is so closely tied to that role.

                                                      I think it’s more charitably viewed as ‘people on the sidelines got enough access to make themselves heard’.

                                                      The other thing I noticed was an inconsistency. Certain words or symbols, esp that were associated with hate, are forever banned from being reused in new situations

                                                      Agree it’s yuck and weird. That said, if you’re going to get hung up on ‘their side are inconsistent’, there hasn’t been a movement in history to meet your criteria.

                                                      1. 8

                                                        This is purely anecdotal, as it’s just my experience on the matter, but what ultimately convinced me to accept the redefinition of gender with respect to sex wasn’t evidence, it was decency. Fundamentally, there’s no particular reason I should take a position on how other people should identify - this goes beyond gender, but that’s a particularly prominent example of the general point. The most effect it has on me is in choosing pronouns, and in my experience most people will be fine with a genuine apology if you make a mistake - even if you make the mistake repeatedly over time, as long as you’re also clearly trying and being respectful.

                                                        Obviously, a person’s sex is of practical importance in a lot of situations; medical treatment, shopping habits (does this person need access to tampons/menstrual pads?), etc., but most of those situations don’t involve me and I can’t think of a single one that can’t sensibly accommodate a decoupling of gender and sex.

                                                        So ultimately, for me, it didn’t come down to an evidence-based, rational decision along the lines of “Well, I’ve been convinced this is how the world actually works”. Instead it came down to recognizing that “how the world actually works” is an absurd concept in a fundamentally subjective topic, so what I really ought to do is just accept that people work differently from one another. My only vehicle for understanding others has always been to trust their self-description, so what’s the harm in applying that here? I’ve got no special perspective that gives me authority over who is or is not appropriately described with certain words, so the decent thing to do is to back off and let people be who they are. And I’ve also got no reason to believe that such a perspective is even conceivably possible.

                                                        1. 2

                                                          Coming back to this later, I think I didn’t word it as well as I could’ve - it’s really easy to read this as saying that anyone who disagrees with me is not being decent (and I got a -1 troll vote that I hope was because of that instead of some other mistake I haven’t noticed).

                                                          I used the word “decency” because it’s the best reflection of how I think about my own ethics, and that’s tangential as hell so I’m not going to waste more words talking about it here. The point I wanted to make was that my take on the subject turns out to be orthogonal to evidence-based reasoning - not that I think you have to agree with me to be a good person. You can have a different ethical code that doesn’t lead you to the same conclusions, and that code could be entirely sane.

                                                    3. 10

                                                      I want three things from a doctor: the same exact medical care as anyone else, an estradiol scrip, and a letter to change my gender on my passport. All of these things are pretty reasonable.

                                                      We shouldn’t have gender on passports, because if it’s referring to biological sex it’s not useful for identifying people and if it’s not then it’s… still not really useful for identifying people. I mean, there’s a photo on there, right? Failing that, people shouldn’t demand to see other peoples’ identification before calling them what they ask to be called. I go by Nora in a lot of places, but my ID says “Leonora”. Nobody shits themselves over that, but for some reason when I had an ID that said “M” everybody was all in a tizzy.

                                                      In any case, it seems pretty reasonable to get such letters so long as they’re needed for people to be respectful.

                                                      Estradiol should be informed consent, and usually is these days. WPATH and other professional organizations recommend this, and doctors who don’t do it are not really doing a good job. It’s not even expensive; you take a blood test a couple times to make sure your kidney’s aren’t fucked up and then cheap pills that millions of cis women are on anyway. No real personal or social downsides to that one.

                                                      Regular medical care should be provided to me regardless of my transgender status, but isn’t always (see: trans broken arm syndrome), in much the same way it’s not always provided to women in general, or fat people.

                                                      The biological model of gender is supported by massive amount of physical evidence which the doctors were educated in and see in their patients everyday. The new ideas of gender identity originated in far-left colleges with basically no hard evidence backing them: just psychological studies (“soft science”) of what trans people believe about themselves and tell them.

                                                      I’m curious what you mean here. What is the “biological model of gender”? Do you mean the idea that gender and sex are the same thing? Sure, that’s fine, but we still need a word for “the internal understanding of one’s body and the way it relates to society’s gender roles”. Let’s call it foo. I’m transfoo. Foo isn’t the same as sex. Etc.

                                                      To put it another way: what evidence would convince you that gender is “really” different from sex? It’s an inherently mental, personal, subjective thing, because it (to a much greater degree than sex) is socially constructed. (Meaning, it’s basically just an agreement between a bunch of people about how people should act. You’re saying it should be based on your junk, I’m saying it shouldn’t, and that’s the only point on which we disagree on this topic.)

                                                      1. 4

                                                        You’re saying it should be based on your junk,

                                                        Yes a lot of these people are acting like if they as a man woke up one morning in a woman’s body that they would just magically be okay with it, and wouldn’t even be upset. IIRC not being bothered by what gender you present means you’re gender fluid, and congrats to you people for being gender fluid, most people aren’t.

                                                        1. 3

                                                          This is actually an extremely good point. I mean, come on, have these people never watched anime? :P

                                                      2. 2

                                                        Exactly. If data scientists want to add a ‘self identified as’ field, they can get a new interesting subset. But they still want the rational physical world for measurements. Could you include people on hormone replacement therapy as a sub-group and look for differences? Absolutely. But it depends on your study and research.

                                                        I wish it weren’t hateful to simply say, “Transgenderism is an ideology,” because it is. Some trans people are fine with being distinct: Like M2F who identifies as a man transitioning or transitioned to a women. But some want to be the other sex, anonymously without any biological distinction. They have to BE the other, in all ways legally and metaphysically. This is mostly fine and no one has a problem with it… except when it comes to unfairness where biological differences are present: like sports. It also applies to medication that might have different dosages based on male/female biological factors.

                                                        It can get heated because it is an ideology and challenge to that is a challenge to orthodoxy. I don’t know what the solution is, but transgenderism is very similar to Catholic transubstantiation; a believe about the metaphysical nature of reality. We’ve never been in this boat in terms of equality and it will be interesting to see where people push this in the coming years.

                                                        1. 17

                                                          transgenderism is very similar to Catholic transubstantiation; a believe about the metaphysical nature of reality.

                                                          Oh you are on SUCH a good track here. I’d like to take this point a little further. Gender itself is a metaphysical belief. So is sex.

                                                          Both sex and gender, as equivalence classes that allow you to categorize all people as one of, say, two types, are broken. They’re fuzzy categories at best, and once you start getting rigorous, you start seeing lots of exceptions. That’s not to say they’re not broadly useful concepts, and they do work for lots of people, but when we’re designing systems that work with sex and gender (for instance, in medicine, or sports!), we should think carefully about what those categories really mean, how they’ll be used, and how they’re going to work for people who don’t fit the boxes neatly.

                                                          For instance, say you’re trying to code people as male or female based on the “rational physical world” of measurements. Do you take chromosomes? Sure. 46XY is male, 46XX is female. I guess we could call 45X female and 45Y male too. What are 47XXX, 47XYY, and 47XXY? 49XXXXY? Y generally induces testicular development, so maybe any copy of Y codes as male. But if you’ve got androgen insensitivity syndrome, some “standard male” 46XY people will have breasts and labia. And some of those 46XX folks with CAH might look outwardly like men. Which soccer team to do you force these people to play on? Who gets prompted for testicular screenings and pap smears?

                                                          We haven’t even gotten into chimerism yet. You know some people have multiple genotypes, right?

                                                          Okay. Let’s back off to genitals. We can measure those, right? If you have testes, you’re male. Unless you’ve had a hysterectomy, or orchiectomy, but you’d know if that happened to you. Well, unless it happened at birth. Turns out that in some cases a doctor guesses, based on some complicated rules and their best judgement, what kind of genitals would be best for you, a newborn infant. Maybe you’re genetically XY, but your penis was small at birth, so they surgically constructed a vagina, removed the testes, slapped an F on the birth certificate, and told your parents you should be raised female. Maybe you never knew about that until getting a gene test in middle age, and then you realized, holy shit, I’ve been male/intersex/trans/etc this whole time. Maybe you’ve got ambiguous genitalia. Maybe you’ve got a vagina, but no uterus, and internal testes instead.

                                                          These things aren’t THAT rare. Roughly one in two hundred people don’t have a “standard” male/female body plan. Chances are you personally know someone like this! About one in a thousand people receive “normalizing” surgery. If you’re a primary care physician, you’ve likely got a couple patients who might need you to have a more nuanced understanding of what sex is. Maybe you need to ask your male patients if they’re considering pregnancy.

                                                          Fine. Let’s decide based on someone’s body’s secondary sex characteristics. Adam’s apples? Facial hair? Voice? Musculature? Height? Breasts? Pelvic geometry? These are all over the place. Slender, wispy people covered in hair. Barrel-chested, high-voiced musclebears with no Adam’s apples. People who fall smack dab in the middle of your female and male coordinates on whatever measurement system you choose. Some of these characteristics are reshapable with surgery, hormones, and practice. Some change with age. But these secondary characteristics are also how we usually code people as male or female. For example, when was the last time you checked to see if everyone walking into the men’s bathroom had a penis?

                                                          And this is just sex! Physical, biological stuff that’s measurable with rulers, blood tests, and CT scans. Gender’s even more spectacular.

                                                          1. 3

                                                            Possibly the best write-up of this particular flaw in the idea of “biological sex is binary” I’ve seen in a while. That’s going in my bookmarks for sure. Thanks :)

                                                            1. 2

                                                              For instance, say you’re trying to code people as male or female based on the “rational physical world” of measurements. Do you take chromosomes? Sure. 46XY is male, 46XX is female. I guess we could call 45X female and 45Y male too. What are 47XXX, 47XYY, and 47XXY? 49XXXXY?

                                                              99.9% of all humans have normal chromosome Karyotype. What you describe are outlies of less than 1% .. You make it sound like all these variations are common when they’re really not. And on top of all of that, almost all people with abnormal karyotypes are sterile. Very few of them can reproduce.

                                                              1. 10

                                                                It’s more like 99%, actually. There are almost a hundred million people out there with some kind of intersex condition - probably more intersex people in the world than desktop Linux and Mac OS X users combined :P

                                                                1. 12

                                                                  99.9% of all humans have normal chromosome Karyotype. What you describe are outlies of less than 1%

                                                                  Sure! XXY is ~1:1000 births. If we were doing research on something with broad interest, like a soda flavor, we might decide that the error we introduce by categorizing those people as male, or female, or removing them from the sample altogether, isn’t significant enough to affect our results, and go on with our lives. It’d be problematic if we needed everyone in a sample of tens of thousands of people to fit exactly one category. Unfortunately, that’s how a lot of our society is built.

                                                                  XXY doesn’t shorten life expectancy much. That means in the US you’ve got ~327,000 people with XXY. A suburban high school with a population of 2000 students has a good chance of having at least one XXY person in their prospective student pool, and will need to decide whether those people are allowed admission (for an all-boys school), and what sports teams (if sex-segregated) they play on. A typical Wal-mart will see a handful a day, and expect them to choose the right bathroom. A suburb of 50,000 could (with sufficient motivation) field an all-XXY football team. A decent-sized airport like Miami International will see ~120 passengers every day whose sex indicated on their ID doesn’t match their karyotype. A major metro area could run a Kleinfelter Kickball league.

                                                                  Let’s imagine you’re like me: a cis man who hasn’t been karyotyped. Maybe you took a 23andme test, and that came back XY, because they don’t report trisomies. There’s a nontrivial chance that you, who grew up believing you were male, who has always been treated like a man, might not be a “biological male”–at least by a strictly genotypic definition. Of course that’s ridiculous. You know you’re a man, and everyone else does too. You don’t get dirty looks for using the men’s room at work. You can hand over your ID at the bar or airport without fear. Nobody called the school board outraged that you were allowed to play on the men’s basketball team. You never had to worry about these things, and you still don’t. That’d be silly. Nobody’s going to find out unless you tell them, and even then, I’m sure they’d understand. Well, most people would. Except for people who believe that “male” means “XY”, and then, well. That’s a different story.

                                                                  That story might involve your marriage being challenged in court–but a neighboring state used anatomy, rather than genetics, to determine sex, so you were able to remarry there and bypass the issue. You may have been asked to use the women’s room at work–only they don’t want you there either, so you wind up having to ask for a special bathroom. The only single-occupancy stall is on another floor, so you take the elevator every time you have to pee, and hope nobody’s using it. You might be stripped of your sports trophies, and banned from competition. You might wind up with a new ID that says F on it. You have to sigh and explain every time you show it, and hope that when you’re pulled over it doesn’t lead to arrest. You haven’t gotten the bank updated yet, and you’re dreading the bureaucratic nightmare that’s sure to follow. Every few weeks, you blunder into a conversation on the internet where someone insists that you aren’t a real man.

                                                                  Think about how that might feel.

                                                                  Now read your posts again.

                                                                  1. 1

                                                                    Klinefelter syndrome. XXY is a chromosome disorder in males. People with XXY chromosomes are classified as male, they present and physically are male, and .. they cannot reproduce. There has never been a recorded case of someone with Klinefelter reproducing.

                                                                    1. 4

                                                                      Okay. I recognize that it’s difficult to go from never having thought critically about sex or gender in your life to Oh Shit All These Things Are Happening??? If it helps, this is a journey we all have to go through. While you’re on it, consider:

                                                                      XXY is a chromosome disorder in males. People with XXY chromosomes are classified as male,

                                                                      You’ve recognized here that “male” and chromosomes are two different things, and that “male” is a category we, as humans, apply to people. It’s also true that people with XXY are usually classified as male, but that’s not always the case. Some people with XXY look completely masculine. Other people with XXY look physically female, with large breasts, soft skin, no beard, etc. Genotype is not always phenotype, which should be the first clue that “biologically male” is an ill-defined concept.

                                                                      they present and physically are male

                                                                      Presentation is a complex phenomenon involving body characteristics, movement, speech, dress, grooming, pronouns, location, cultural roles, etc. These things usually go together in patterns we think of as “male” or “female”, but they don’t have to. Most people with XXY present as men, but not all do! As it turns out, I have a friend with XX+XY (though she’s not sure if that’s via trisomy, a more exotic aneuploidy, mosaicism, etc) who presents in basically all respects as a woman. She’s hella cool. It’d be weird to declare her male.

                                                                      Being “physically male” is a fuzzy concept for all the reasons I outlined above.

                                                                      and .. they cannot reproduce. There has never been a recorded case of someone with Klinefelter reproducing.

                                                                      I don’t think this is relevant, and I’m not sure why you’re bringing it up, but, uh… you know this isn’t true, right? People with Klinefelter do reproduce, both unassisted and with the help of technology. Low fertility is a common consequence of XXY, but it doesn’t render every person sterile.

                                                      3. 3

                                                        That is not the full sentence, though. Since you mention reading comprehension, I would suggest interpretating any alarming sentence fragments not in isolation, but in the context of the full sentence (and the rest of the piece). The full sentence is this:

                                                        There’s no test that you give someone to determine they’re “actually” trans, unless you’re a doctor, or a neuroscience researcher, or a bigot (but I repeat myself).

                                                        Seeing the full sentence this reads like a syllogism: (people who administer) transness tests are bigoted, doctors & neuroscientists administer transness tests, therefore doctors and neuroscientists are bigots. Still an overgeneralisation, but a far cry from ‘doctors and neuroscientists are bigots’ on its own. So, ya, my guess is that probably you misunderstood the author’s intent?

                                                        1. 2

                                                          Ohhh, the irony… It would be funny if it wasn’t so pathetically pervasive.

                                                          1. 2

                                                            It probably refers to how the medical discipline is policing trans bodies and not even trying to renounce its role as gender gatekeepers.

                                                          1. 9

                                                            Hey, mods? For the last five years or so, I’ve titled these articles “Jepsen: [Database Name] [Version]”. I’ve been informed that that’s no longer an acceptable title for Jepsen reports, because “Jepsen” is also the name of the site. Can I get a policy clarification here? Do I need to make up a lobste.rs-friendly title that doesn’t include the word “Jepsen” for each submission from now on?

                                                            1. 7

                                                              The story submission guidelines state, in part: “Please remove extraneous components from titles such as the name of the site, blog, section, and author.” The story was originally submitted with the story title “Jepsen: YugaByte DB 1.3.1” from the domain jepsen.io. Given that the site name appears in the story title, and consistent with the story submission guidelines, I replaced “Jepsen: “ with “Testing “ as the best operational description of the article I could find.

                                                              It has since been brought to my attention that Jepsen is also the name of a Clojure library. As a site name, the story submission guidelines are clear. As a library or tool name, they are mute. As Jepsen is both, but given the existence of an interpretation of the article title that does not violate the site guidelines, I have restored the story title as originally submitted.

                                                              1. 4

                                                                It has since been brought to my attention that Jepsen is also the name of a Clojure library. As a site name, the story submission guidelines are clear. As a library or tool name, they are mute. As Jepsen is both, but given the existence of an interpretation of the article title that does not violate the site guidelines, I have restored the story title as originally submitted.

                                                                This is such a wacky justification! Somewhere, a lawyer smiles without knowing why.

                                                                But I like the “Jepsen: WebscaleDB 0.1.1-alpha2” titles, so I’ll take it.

                                                                1. 3

                                                                  Absent a Damas-Hindley-Milner typechecker for story titles, it’s just us manually splitting hairs to try to keep up their signal:noise ratio. I was also surprised by Jepsen being the name of the tool used for the analysis as well as the project and consulting company I knew about. I can only ask that @aphyr disambiguate with Hungarian Notation - perhaps he’d like to adopt Jepsen_clj, Jepsen_blog, and Jepsen_LLC so we can more efficiently mangle his titles in the future.

                                                                  1. 5

                                                                    I like to think the usage in the submission title here is a fourth meaning of “Jepsen”, being a proper name not of the blog itself but of the kind of article the blog contains, i.e. “a Jepsen of YugaByte DB”

                                                                2. 3

                                                                  Thank you. For what it’s worth, I prefer to see Jepsen in the title, because in my mind, it adds a ton of credibility for the content.

                                                              1. 7

                                                                @aphyr Would you see value in community funded analysis of PostgreSQL?

                                                                1. 7

                                                                  Oh absolutely. I’d love to do PostgreSQL at some point!

                                                                1. 4

                                                                  thank you for your work @aphyr.

                                                                  You are like FDA, but for distributed databases :-) (and without the ‘F’ ).

                                                                  As a suggestion to ‘consumerize’ your work more, if you will. Is to publish a news letter or a blog that will continuously maintain a table listing what tests were performed, what databases participated (version/vendor), and result within each cell.

                                                                  Since your tests are standardized, having that comparative view, I think would be very useful.

                                                                  1. 8

                                                                    I don’t know a good rubric for comparing results yet, but I do maintain a table of analyses here: http://jepsen.io/analyses

                                                                    1. 3

                                                                      It would be great to see some of the cloud provider’s managed databases.

                                                                      1. 4

                                                                        That’s gonna be tougher–I don’t have a good way to inject faults into those systems. We can test the happy case, though!

                                                                  1. 23

                                                                    I absolutely do not want to be seen as being in the same camp as the low-life scum on the internet that think it’s OK to be a white nationalist Nazi, and have some truly nasty misogynistic, homophobic or transphobic behaviour. […] And those people were […] making [me] look bad.

                                                                    Makes sense.

                                                                    I’m anti-CoC in principle but pro-CoC in practice. Why? Because of those “truly nasty” folks. I’m pro-meritocracy–in principle! But, the word is just a dog whistle now. I am truly tired of randos on the street trying to explain to me why the internet is terrible. Is this famous person publicly distancing himself from them going to help?

                                                                    1. 17

                                                                      I’m pro-meritocracy in principle too, but I think the belief that an organization is meritocratic can and is used to justify a lot of ugly behaviors. Better to have meritocracy as an ideal and work towards it than ever claim to actually be, because it leaves room cognitively to recognize and correct when it’s not true.

                                                                      1. 19

                                                                        Meritocracy is not a fucking dogwhistle. I know I’ll get down voted for saying ‘fuck’ but it’s worth it. The emphasis is worth it. What you said is complete and utter nonsense.

                                                                        Meritocracy is a great thing. The people calling it a dog whistle are the sort of people that get funded by outreach programmes to add ‘const’ to a hundred lines of code in the linux kernel and call themselves kernel developers then get offended when people suggest that’s not very good use of money. The people calling it a dog whistle are incompetent people and people that listen to incompetent people.

                                                                        A ‘meritocratic’ country inevitably results in a country that lets poor people die on the street. But a free software project is not a country. The free software community is not responsible for providing a social safety net. Meritocracy - the idea that the people that run things and make decisions should be those with technical merit - is the only way free software works.

                                                                        I’ve been part of open source projects run by non-technical people that picked and chose leaders and decision makers based on how much they were liked and how long they had been around and how active they were on IRC. It’s a broken model. If you’re interested in free and open source software projects in the sense that you want to actually see them develop the software ever, meritocracy is code for ‘get on with writing code and leave the politics for the voting booth’.

                                                                        1. 14

                                                                          Wikipedia’s Criticism of meritocracy section is a good overview of why a true meritocracy is difficult or impossible—but I don’t think you’re actually arguing for meritocracy here. It seems like your main point is that (at least in volunteer efforts like Free Software projects) work should be allocated to the people best able to perform that work, otherwise nothing will get done. That seems like a reasonable approach, and I’ve sometimes heard this described as a do-ocracy, where things get done by the people with the skill, motive and opportunity to do them.

                                                                          I don’t think it’s strictly-speaking a meritocracy, though, since getting something done requires motive and opportunity as well as skill. Maybe the person with the most skill doesn’t have the opportunity, or they’re just interested in other things at the moment and don’t have the motive. Maybe a particular achievement requires a variety of skills, and the person who gets something done has many weak skills, even though there are many people who are much stronger in any individual skill. I’d be willing to believe that in Free Software projects, the people who get the most done are not the most technically skilled, making them the opposite of a meritocracy.

                                                                          1. 4

                                                                            That wikipedia articles goes through multiple definitions of meritocracy – but the first one you come into contact with reads

                                                                            … should be vested in individuals on the basis of talent, effort, and achievement, rather than factors such as sexuality, race, gender, or wealth.

                                                                            Talent, effort, AND achievement, which is what I think most people resonate with when they see the word meritocracy. Achievement is the doing of tasks, it takes effort and talent. I never considered a meritocracy to be based solely on skill.

                                                                            I’ve sometimes heard this described as a do-ocracy, where things get done by the people with the skill, motive and opportunity to do them

                                                                            Whom, I would think in the context of the project would be those who merit praise and control? I guess it is a question of order to some degree – do you get assigned something due to your merit, or do you get merit based on what you have done? I suspect many people think in terms of the latter not the former.

                                                                            Maybe the person with the most skill doesn’t have the opportunity, …

                                                                            Is this going with the definition that merit is only skill (talent) and ignoring effort and achievement? Akin to the referenced “1956 A. Fox in Socialist Comm. May 13/1 The ‘meritocracy’; the society in which the gifted, the smart, the energetic, the ambitious and the ruthless are carefully sifted out and helped towards their destined positions of dominance.”?

                                                                            I think in open-source, merit in terms of control tends to follow activity, not the reverse. Additionally, if anyone is trying to help people towards their destined positions of dominance – wouldn’t that be people trying to promote certain classes of people?

                                                                            1. 2

                                                                              I guess it is a question of order to some degree – do you get assigned something due to your merit, or do you get merit based on what you have done? I suspect many people think in terms of the latter not the former.

                                                                              In a system where you don’t get authority due to your merit, that’s not really a “merit-ocracy”, is it? The name for “you get merit based on what you have done” is just “remuneration”.

                                                                              A better model would be a cycle: you attempt a small thing, you earn merit (by your demonstrated effort and achievement) and you get to attempt a bigger thing to earn more merit. But everybody (by definition) starts out with zero demonstrated effort and zero achievement, so a meritocracy needs to be bootstrapped with some other principle.

                                                                              Since (in a meritocracy) you need merit to earn merit, merit is an inflationary cycle, and a small initial advantage can become a huge long-term advantage. Whatever principle is used to bootstrap the meritocracy can therefore have a greater effect on society than the meritocratic principle, even over the long term. And if “merit” isn’t the most relevant attribute, it seems misleading to describe the result as “meritocracy”.

                                                                              1. 1

                                                                                “But everybody (by definition) starts out with zero demonstrated effort and zero achievement, so a meritocracy needs to be bootstrapped with some other principle.”

                                                                                I don’t think that’s true with code. You get merit when you write useful code in a meritocratic system. You did it here, there, wherever, and you’re a coder now. The more output or impressive designs, the more merit.

                                                                                I don’t think a bootstrapping phase is needed if it’s really a meritocracy. It might be if it’s not. ;)

                                                                                1. 2

                                                                                  My point is, you can’t have a true meritocracy where great merit is built on lesser merit, like great turtles stacked on lesser turtles, because eventually the turtles have to be stacked on something. You seem to be saying that one project can start stacking its turtles on some other project’s turtle stack, which while true, doesn’t invalidate my point.

                                                                                  Once you get to the point that you’re comfortable writing code and putting it online for the world to see, congratulations, you’ve gotten past the bootstrap principle and hopefully you can operate by the meritocratic principle from then on. But that doesn’t mean that the bootstrap principle doesn’t exist, or that it has anything to do with merit.

                                                                                  1. 1

                                                                                    That makes sense.

                                                                            2. 3

                                                                              This resonates with me a lot. I suggest reading the Walkaway from Cory Doctorow to anyone interested as it elaborates on the difference between just doing and a meritocracy in one of the earlier chapters.

                                                                              I believe that empowering as many people as possible to contribute in a useful way is better than making it possible for a few competent ones to order the rest around. And by empowering I mean providing processes, tools and communications accessible to anyone willing to volunteer their time.

                                                                            3. 9

                                                                              people that get funded by outreach programmes to add ‘const’ to a hundred lines of code in the linux kernel and call themselves kernel developers then get offended when people suggest that’s not very good use of money

                                                                              Did that actually happen?

                                                                              1. 1

                                                                                X, age 22, of India, has had more than 340 patches accepted into the Linux kernel – an accomplishment that contributed in no small part to her receiving one of two Linux Kernel Guru scholarships from The Linux Foundation.

                                                                                X served as an Outreachy intern earlier this year, focused on the Linux kernel, where she worked on securing the kernel from surface attacks by making the kernel structures read-only.

                                                                                Name changed to avoid any suggestion I’m trying to publicly shame her or anything like that. But yes, it did actually happen.

                                                                                1. 4

                                                                                  served as an Outreachy intern earlier this year, focused on the Linux kernel, where she worked on securing the kernel from surface attacks by making the kernel structures read-only.

                                                                                  I looked at the kernel change log. That’s not a trivial project at all. The modifications are simple, but it required reading a lot of kernel code to see what could be done and it looks entirely useful. Pretty good for an intern project.

                                                                                2. 1

                                                                                  To me, it’s much more common to see libertarian theorists who glomm on to others work, hustle funding from not too sharp corporate programs, and portray themselves as superheros.

                                                                                  1. 0

                                                                                    Of course not. It’s a routine strategy of CoC opponents to invent fantasy grievances and use slippery slope arguments in order to excuse, ignore, or detract from the real challenges a CoC is trying to address.

                                                                                    1. 4

                                                                                      I really wish they’d knock that off if they’re doing it. It’s much easier to link to the real actions of the person, Ehmke, who wrote Contributor Covenant and pushed for CoC’s. She wrote it specifically to do things like that. In there, she’s slamming a maintainer and project pushing them to reinforce her beliefs and practices about how the project should be run… because that’s what the CoC is designed to do. Not just make people be nice, not total assholes, or whatever. In that example, they did the following:

                                                                                      1. Show their belief that a person expressing different, political views in any medium is to be ejected from all mediums these people wield influence on, including their own software project.

                                                                                      2. They demand a key contributor of the project that its survival might depend on be ejected with nothing in return. They expect everyone they target like this to comply regardless of the cost. The maintainer challenges them to pick up their slack contributing to the project. They show no intent to give anything in return for what they demand.

                                                                                      3. They hit it hard all at once as a crowd ganging up on their targets, relentless, and increasing aggression as time goes on. Imagine a crowd showing up at your door unannounced yelling insults at you telling you all the changes you’ll make in your social circle or hobbies to comply with their beliefs. They want you to tolerate and comply with that situation. Depending on the location, such people would get told to get lost, be arrested, or get shot by homeowner. These people writing and pushing the CoC think it’s mandatory to do it and other accept it.

                                                                                      4. The CoC pushers show true colors when insinuating the maintainer supports child rapists. Started with a poisoned question followed by some shaming. When I saw that, I dismissed them now and forever if they stay on same path since that’s so low it’s sickening to watch. Not to mention trivializes horrible thing. Even most sophist debaters won’t connect opponents to child rapists. These people are both ideological and “win at all costs” in a way worth stopping fast.

                                                                                      5. Toward the end, they get into the insults and stuff showing they don’t care about offense, inclusion, etc so long as opponents have different political philosophy. They talk of burning bridges but to who.

                                                                                      They’re usually a lot sneakier so that people can say what you just said. In that case, they went full on thinking so many people were like them and they were so much better that their plan would work with no problems. Maintainer was expecting an attempt like this, stayed in constant No mode without fighting them, and that strategy outed them better than any other. This behavior, not being transgender or trying to make peaceful communities, is why Ehmke takes so much shit from people on the net, in Github, etc. It’s also why a tool designed for such political attacks and subversion, the Contributor Covenenant CoC, should never be adopted by diverse projects if there’s even a chance such people will be enforcing it or have significant sway. I added diverse since their beliefs seem uncommon to both majority and (in my area) minority members. Latter may or may not be true in other locales, but these people don’t care.

                                                                                      People should know their goal is to silence and eject everyone that disagrees with them even on their own Twitter account speaking personal opinion. They say they do this to prevent offense and be inclusive. Yet, they’re willing to maximize offense and exclusiveness to anyone that disagrees with them saying the end justifies the means. Even minority members disagreeing are suppressed as deluded by people with such politics. So, going with them is supporting a radical version of leftist politics willing to censor everyone from liberals to conservatives that don’t agree with them, even if otherwise civil. That’s unacceptable for an inclusive, democratic, and/or meritocratic project.

                                                                                      And to people that think it’s inherently necessary: we disprove the theory of people like Ehmke here on Lobsters regularly with better moderation. There’s rarely bans. The group-oriented censorship that exists is mostly collapsed comments that still allows exploration of unpopular views. The moderators that disagree tolerate it if the person is still civil. The people that disagree a lot sometimes still are helpful to each other. So surprising if the theory is we can’t have a diverse, productive, and enjoyable community without CoC’s and moderators strictly enforcing radical, leftist politics.

                                                                                      1. 2

                                                                                        I was actually about to respond to your points, when I realized how deftly you had deflected away from mine, first by pretending to agree, then by denying [“if they’re doing it”—as if it hadn’t been right there in the grandparent post], and finally pivoting to another topic altogether. Specifically, no one claimed that CoC proponents have never acted in bad faith.

                                                                                        So I won’t engage with that. Nice try though!

                                                                                        1. 3

                                                                                          I said the main CoC author and proponent designed it based on her group’s faith, used it to enforce that in many cases, forced others to adopt it with vicious tactics, and continues lie in various articles about its purpose and where the resistance is coming from.

                                                                                          There’s no try from me so much as showing the CoC inventor using the CoC to do what it’s designed to do with her scheming more visible. Especially that it’s quite different goals and results versus what she tells many projects and companies about when pushing its adoption. That difference is important.

                                                                                          1. 1

                                                                                            Great! Go tell it on the mountain! Or the moon for all I care.

                                                                                            1. 2

                                                                                              “We are concerned about racist/sexist/immoral behavior in your project/company.”

                                                                                              “Great! Go tell it on the mountain! Or the moon for all I care.”

                                                                                              Still accomplishes same thing. Would you want you’re political opposites running projects to stick to those kind of replies?

                                                                                          2. 3

                                                                                            Refusing to engage with someone because they didn’t either completely disagree with you or completely agree with you is arguing in bad faith. I have downvoted you and marked you as a troll. I think it’s bad form to downvote without giving an explanation so that’s why I’m giving you one.

                                                                                            1. 2

                                                                                              Hi, I upvoted you because who gives a shit and I’m still not going to engage with nick

                                                                                          3. 1

                                                                                            All human projects are rift by politics, ambition, honest and dishonest differences of opinion, mistakes, anger, and all those other messy human qualities. Get used to it.

                                                                                            1. 1

                                                                                              In another comment, you expressed skepticism about some conspiracy by people to push their agenda with CoC’s. You talked like they’re a neutral tool designed just to get rid of bad behavior we’d all agree on. Many of you either buy into that disinformation or spread it. So, I linked to example of author using her work to do exactly the things people worried about that you dismissed as hypothetical.

                                                                                              Oh I know people have political differences and it gets messy. Im one of a few people defending our right to on the forum. Then, there’s sneaky, political activists that tell a pile of lies about their goals, try to force their politics on others, and (surprise) pretend they are victims and/or werent doing anything at all when people resist that. Im shining the light on those scheming, lying pricks.

                                                                                              In case you didnt know, exposing the lies of political aggressors is part of accepting and operating in a world of politics, their ambitions, and their dishonest differences of opinion.

                                                                                              1. 3

                                                                                                I have never suggested or argued that human beings are free of bad motives or agendas. But I’m trying to point out that these CoC’s are not being applied to previously angelic communities of disinterested technical enthusiasts - but rather to standard human groups that are already political, influenced by money, full of people who are prejudiced and motivated by all sorts of not necessarily great ideas. I was an early participant in Linux. I heard open racism and sexism plus a lot of corporate bullshit and worse. It was not shocking

                                                                                                1. 1

                                                                                                  That’s a fair point. I’d still oppose them without modification but they would have new form. Trading one type of politics and risk for another which may be better.

                                                                                          4. 2

                                                                                            Of course not.

                                                                                            Well it did actually literally happen. Would you like to apologise to calling me a liar?

                                                                                            I’d suggest that if anyone is ‘inventing fantasy grievances’ it’s the people that come up with policies like Netflix’s new ‘you may not look at any coworker for more than 5 seconds at a time’ policy. The ‘real challenges’ a code of conduct is trying to address don’t exist. Having rules is one thing. Online communities have always had rules. Nobody has a problem with having rules. What people have a problem with is codes of conduct, because ‘code of conduct’ means ‘Americanised overly-political unnecessary rules’.

                                                                                            1. 2

                                                                                              “Her goal is to become a full time kernel engineer after completing this current project.” [emphasis added]

                                                                                              Now did you accurately represent that part in your comment? Or did you willfully omit or misrepresent that part?

                                                                                        2. 8

                                                                                          I think you had already in mind what you wanted to say before reading what @sebboh wrote and went ahead.

                                                                                          He is not against any of those just how the words are just thrown around.

                                                                                          It’s like one being clearly against being a douche to others but still not liking how certain pro-CoC people are behaving.

                                                                                          1. 7

                                                                                            “It’s like one being clearly against being a douche to others but still not liking how certain pro-CoC people are behaving.”

                                                                                            Describes me pretty well. Keep it civil but tolerate differences.

                                                                                          2. 12

                                                                                            What part of “I’m anti-CoC and pro-meritocracy” did you miss, milesrout?

                                                                                            I presume you’re mad at someone else and you’ve simply misidentified me as them. I forgive this. It’s been a long and maddening war; we’re all stressed.

                                                                                            Now please sit out while somebody answers my question. Will this instance of a famous ubergeek publicly distancing himself from those truly hateful folks do anything to help end the war? Can we yet move beyond Red vs. Blue? If not, what will it take?

                                                                                            UPDATE: Sorry, I hit ‘post’ too soon. I meant to also say that the word meritocracy is being used as a dog whistle “now”. As in, ever since this current Linus news item and the response. “Used as”. I didn’t start it, I’m just reporting what I see.

                                                                                            1. 5

                                                                                              I don’t think meritocracy is used as a dog whistle by anyone though. That’s kind of my point.

                                                                                              Sorry that my post was a bit aggressive. This whole CoC stuff just aggravates the crap out of me.

                                                                                              1. 8

                                                                                                This whole CoC stuff just aggravates the crap out of me.

                                                                                                In all seriousness: why?

                                                                                                1. 12

                                                                                                  Of course it is used as a dogwhistle. People have been using meritocracy or similar to justify privilege since the first cavepeople started killing each other. It’s super common for people who have unearned privileges, often unjust privileges, to angrily insist that they got what they have through hard work, God’s will, superior morals or intellect, better heredity, racial superiority (or inferiority of the unfortunate other people) or anything at all. It doesn’t matter whether you are justifying the English ruling class in 1100AD, or google programmers last week, or Tsutsis or Serbians or whaever - it’s all a shoddy bullshit effort to justify the unjustifiable.

                                                                                                  1. 3

                                                                                                    In your definition of meritocracy is it about innate ability, or is it about demonstrated skill/value?

                                                                                                    My point being, someone of privilege might have the ability to spend all their time learning, lets say polo. They become the worlds best polo player. Another human, who through the genetic and birthplace lottery is the theoretical best polo player ever but due to circumstances of birth never even sees a horse.

                                                                                                    The privileged person is the demonstrated in real life best polo player, they have actually done it. The other person while having more raw ability through the genetic lottery did not accomplish anything (in terms of polo). Is this a failure of a meritocracy, should it somehow have found the person with the most potential? Is this person who gets praise as being “the best polo player” unjustified?

                                                                                                    1. 4

                                                                                                      You can have merit (e.g. acquired through skill and work) and also have unearned privilege. This is not a binary. However, if you are a really excellent, meritorious, polo player thanks to some combination of your skills and your parent’s wealth, you should try not to whine about how you are being discriminated against if the polo league invests in adding training opportunities for less well to do people.

                                                                                                      1. 2

                                                                                                        The implication from your response is that you think the merit is based on accomplishment, not potential. I don’t intend to misrepresent you, is that correct? Because if you believe it is accomplishments that create merit, then wouldn’t it follow that regardless of privilege those who accomplish the most merit the most acclaim, money, etc?

                                                                                                        You could say most athletes at the top level are genetically privileged, they have a natural top level that exceeds that average person due to simple physics. They would likely claim they came to the top of their sport via a meritocracy, they were simply better than others, would you disagree with them?

                                                                                                        You could easily extend this to wealth, a person with the best coaches and training might have an advantage beyond their genetics over another human being in terms of sport $X. They demonstrated the ability to win, they did so against other competitors – did they rise through a meritocracy?

                                                                                                        1. 3

                                                                                                          You’re oversimplifying to avoid the mess. If law/custom prevents women from becoming mathematicians, and you are a man who is one of the top mathematicians of your era, your accomplishments are a mixture of work/ability and privilege. You can’t claim to have risen to the top of the field on your own hard work/smarts, when half the population is prevented from competing. Your field cannot be fairly called a meritocracy. When orchestras started doing blind auditions, they discovered that they had not been, as they presumed, promoting on merit alone.

                                                                                                          1. 3

                                                                                                            I am not oversimplifying to avoid the mess, I am trying to reach understanding. We generally try to reach understanding via simplification down to root principals.

                                                                                                            So by your definition of meritocracy – a field can only be a meritocracy if it has 100% of the population able to participate? Basically 100% EoO (Equality of Opportunity). It would follow then that you believe meritocracies do not exist – as that has never occurred, nor do I think it is likely to ever occur.

                                                                                                            It seems to me that your point is more forceful and clear if you simply start from “meritocracies don’t exist” – they aren’t a dog-whistle, they are a straight up lie. Anyone referencing them is lying to further their own interests, or defend their status.

                                                                                                            1. 3

                                                                                                              I’d generally agree with the claim that “meritocracy cannot exist”; messy real-world issues mean that it can only be approximated.

                                                                                                              I’d go further, and say that calling something a meritocracy is an attempt to pretend that those messy real-world issues are not at play.

                                                                                                              1. 3

                                                                                                                And I would say that is both clear and fairly compelling. I think calling it a dog-whistle is a disservice to that argument.

                                                                                                                1. 3

                                                                                                                  I remember back in the late ’90s when Ira Katznelson, an eminent political scientist at Columbia, came to deliver a guest lecture to an economic philosophy class I was taking. It was a great lecture, made more so by the fact that the class was only about ten or twelve students and we got got ask all kinds of questions and got a lot of great, provocative answers. Anyhow, Prof. Katznelson described a lunch he had with Irving Kristol back either during the first Bush administration. The talk turned to William Kristol, then Dan Quayle’s chief of staff, and how he got his start in politics. Irving recalled how he talked to his friend Harvey Mansfield at Harvard, who secured William a place there as both an undergrad and graduate student; how he talked to Pat Moynihan, then Nixon’s domestic policy adviser, and got William an internship at The White House; how he talked to friends at the RNC and secured a job for William after he got his Harvard Ph.D.; and how he arranged with still more friends for William to teach at UPenn and the Kennedy School of Government. With that, Prof. Katznelson recalled, he then asked Irving what he thought of affirmative action. “I oppose it”, Irving replied. “It subverts meritocracy.”

                                                                                                                  Attributed to Harry Hopkins.

                                                                                                                2. 1

                                                                                                                  You can say it can only be approximated about the goals of common alternatives to meritocracy as well. They’re all models we might strive for which will have failures or exceptions.

                                                                                                                  The model did have successes, though, like we saw with blind auditions. That’s a performance-focused technique that ignores people’s differences entirely. That’s what got more women promoted in Navy. Gapjumpers also claimed success with that model.

                                                                                                                  So, meritocracy seems like it can work if it’s blind to our differences. It’s the conscious/unconscious biases that seem to screw everything up.

                                                                                                                  1. 2

                                                                                                                    I agree that creating (or moving towards) a blind meritocracy is generally going to lead to better outcomes. It’s the claim that a group has already arrived there which I object to.

                                                                                                                    The pro-affirmative-action crowd argue that its not possible to get blind enough, so we need active counter-steering. That’s where I think principled debate is possible.

                                                                                                                    1. 1

                                                                                                                      Another great counterpoint. :)

                                                                                                    2. 3

                                                                                                      Me too and me too. Gah. I was still thinking about this thread after I signed out and I quickly realized that ‘dog whistle’ is certainly not a good term to describe what I observed… Sorry about that!

                                                                                                      A dog whistle is, for example, when somebody is speaking to a group of people and they use a phrase that carries extra meaning for a subset of the group and the speaker does it deliberately to “get away with” saying something they aren’t willing to say outright.

                                                                                                      What I see happening with the Linus-takes-a-break conversation is: folks picking up the thread and twisting it around so that it looks as if the pro-CoC crowd has committed some heinous manipulation. But that is not what Linus said happened. In fact, he said “I absolutely do not want to be seen as being in the same camp as the low-life scum […]”. Oh, ‘spin’?

                                                                                                      Ok, so I should have said: I’m pro-meritocracy–in principle! But, the idea is being subjected to PR-style ‘spin’ now, as if to force good people to choose between adopting a CoC and opposing meritocracy. Damn the heathens who have constructed this probably false dichotomy!

                                                                                                  2. 11

                                                                                                    Meritocracy is not a fucking dogwhistle.

                                                                                                    It may interest you to know that the term “meritocracy” was invented to be precisely that: a signal that the speaker falsely believes their position of power to be justly earned, as opposed to the product of social stratification: https://www.theguardian.com/politics/2001/jun/29/comment

                                                                                                    1. 2

                                                                                                      To be fair – words once unleashed are defined by society and not the original authors. I am not sure the average person thinks of meritocracy in terms of what it was coined to mean. Lump it in with hacker, awful, literally, girl, and tons of other words that have completely changed meaning over the years.

                                                                                                      Possibly “meritocracy” is a poor word to be used if two people will sincerely read it exceptionally differently.

                                                                                                      1. 1

                                                                                                        The original definition of racism was a practice done to Native Americans that most minority members have never experienced. Do you think they similarly should stop redefining the word when saying they were victims of racism? Or are terms allowed to evolve?

                                                                                                      2. 5

                                                                                                        I’m pretty sure you’re allowed to drop f-bombs on Lobsters.

                                                                                                        1. 22

                                                                                                          You’re also allowed to say fuck.

                                                                                                        2. 8

                                                                                                          The word “dogwhistle” is a dogwhistle. It’s the magical sauce you sprinkle on somebody else’s words when you want to pretend they mean the most awful thing you can imagine, rather than what they actually said.

                                                                                                          1. 2

                                                                                                            People who are making really vile arguments often try to disguise them and then they and others will argue that as long as you don’t openly advocate something terrible, you are not.

                                                                                                            1. 1

                                                                                                              Now that you mention it, the radical Leftists do the same thing. Their real, vile argument is all people should be forced to conform to their narrow views by any means necessary and in perpetuity. Then, they try to disguise it when pushing CoC’s by talking about goals and offenses nobody would disagree with.

                                                                                                              The bad folks in very different groups are a lot alike as usual.

                                                                                                      1. 5

                                                                                                        Another great/thorough writeup. And kudos to the dgraph team (and all of the other previous jepsen customers), for putting their money where their mouth is to have their work publicly torture tested like this.

                                                                                                        @aphyr, 3 questions if you don’t mind the tangent:

                                                                                                        • Any chance on a FoundationDB analysis?
                                                                                                        • Are there a lot of jepsen users out in the wild? I imagine some bigger places have engineers that have a need/capability of using it within their own organization, but I don’t know what kind of feedback (if any) they give you.
                                                                                                        • Any thoughts on integrating jepsen with more formal methods? Any engineering pros/cons would be insightful.


                                                                                                        1. 7

                                                                                                          Thank you! To answer your questions…

                                                                                                          1. There’s no FDB analysis planned; they haven’t approached me, and since I just moved and took a couple months off, I really need to focus on taking paying gigs and rebuilding funds. My next client is all lined up though, and I should have more results to show in winter. :)

                                                                                                          2. It’s hard to say! I get PRs from maybe 5 active orgs, and I know of… maybe a dozen orgs who use it independently? I’ve also trained… maybe 150 people in writing Jepsen tests, but I don’t necessarily know whether those folks went on to use Jepsen at their orgs internally, adapted the techniques to their own test suites, or moved on to other things. I think the techniques are more important than the tool itself, so even if folks aren’t using Jepsen itself, I’m happy that they’re doing more testing, fault injection, and generative testing!

                                                                                                          3. I have sort of a “part of this balanced breakfast” take on Jepsen–it exists on a spectrum of correctness methods: normal proofs, machine-checked proofs, model checking, simulation testing, the usual unit & integration tests, Jepsen-style tests, internal self-checks, production telemetry, fault injection/chaos engineering, and user reports. In the early design phase, you want provable algorithms, but complexity might force you to give up on machine-checked proofs and move to model checking; model-checking covers weird parts of the state space but isn’t exhaustive, so it’ll miss some things. The map is never the territory, so we need simulations and tests for individual code components and the system as a whole, to verify that each piece and the abstraction boundaries between them hold up correctly. As you move to bigger tests, you cover more system interactions, but the state space generally explodes: larger tests explore less of the state space. Jepsen’s at the far end of that testing continuum, looking at all the interactions of a real production system, but only over short, experimentally accessible trajectories–a simulation test, like FDB does, is going to cover a lot more ground in the core algorithm, but may not catch bugs at the simulation layer itself or in untested components, e.g. a weird interaction between the filesystem and database which wouldn’t arise in an in-memory test. And Jepsen is specifically constrained to simple, testable workloads; it’s never gonna hit the data or request volumes, or query diversity, that real users will push at the system–that’s why we need user reports, telemetry from production, self-checks, etc.

                                                                                                          There’s a lot of “formal methods” in Jepsen; every test encodes, more or less explicitly, an abstract model of the system being evaluated. We take a range of approaches for performance and coverage reasons, so some actually involve walking graphs of state spaces, and others are just checking for hand-proved invariants. Developing new and faster checkers is a great place to apply your formal methods knowledge, if you’re looking to contribute!

                                                                                                          1. 1

                                                                                                            re FoundationDB. I wanted to see that, too. He had good reason for not doing it described here. In that thread, he said he hasn’t tested it because “their testing appears to be waaaay more rigorous than mine.” Still might be good for independent replication, though. Plenty of scientific papers look like they have a lot of rigor until you find that they missed this, used that incorrect algorithm, or just made stuff up for fame or fortune.

                                                                                                            I say that as someone who was Wow’d by FoundationDB. Hopefully, Jepsen just confirms it was as good as it appeared. If not, people get to fix any problems he finds. It’s all-win scenario unless he finds a problem they can’t fix somehow.

                                                                                                            1. 3

                                                                                                              That was based on a phone conversation I had with one of the FDB team members–they were doing a bunch of tests, like hardware faults and simulation tests, that weren’t really feasible for Jepsen because a.) I didn’t have custom hardware, and b.) simulation testing has to be built into the database code itself, and Jepsen takes a black-box approach. FDB also spun up their own Jepsen test, but I can’t tell you how deeply they explored there.

                                                                                                              Then FDB got eaten by Apple, and fell off my radar–but I’m happy it’s re-emerging now! We don’t have any plans to work together right now, and I’ve got my hands full with other clients, but I’d be happy to work on FDB tests in the future. :-)

                                                                                                          1. 8

                                                                                                            Massive kudos to this guy for not putting up with this SJW madness. I wish him all the best!

                                                                                                            We at suckless are heavily opposed to code of conducts and discriminatory organizations of any shape or form.

                                                                                                            1. 13

                                                                                                              Suckless takes a similarly principled stand against runtime config files.

                                                                                                              1. 11

                                                                                                                How does suckless oppose discrimination?

                                                                                                                1. 15

                                                                                                                  It’s very simple. Any non-technological matters during software development move the software away from its ideal form. Thus, to make your software suck less, you only take the best developers no matter what race, gender, heritage, etc. these persons have.

                                                                                                                  We do not believe in equal status (i.e. e.g. forcibly obtaining a 50/50 gender ratio), as this immediately leads to discrimination. We do however strongly believe in equal rights, naturally. You also naturally cannot have both.

                                                                                                                  1. 109

                                                                                                                    Any non-technological matters during software development move the software away from its ideal form.

                                                                                                                    Suckless makes a window manager: a part of a computer that human beings, with all their rich and varying abilities and perspectives, interact with constantly. Your choices of defaults and customization options have direct impact on those humans.

                                                                                                                    For example, color schemes determine whether color-blind people are able to quickly scan active vs inactive options and understand information hierarchy. Font sizes and contrast ratios can make the interface readable, difficult, or completely unusable for visually impaired people. The sizes of click targets, double-click timeouts, and drag thresholds impact usability for those with motor difficulties. Default choices of interface, configuration, and documentation language embed the project in a particular English-speaking context, and the extent to which your team supports internationalization can limit, or expand, your user base.

                                                                                                                    With limited time and resources, you will have to make tradeoffs in your code, documentation, and community about which people your software is supportive and hostile towards. These are inherently political decisions which cannot be avoided. This is not to say that your particular choices are wrong. It’s just you are already engaged in “non-technical”, political work, because you, like everyone else here, are making a tool for human beings. The choice to minimize the thought you put into those decisions does not erase the decisions themselves.

                                                                                                                    At the community development level, your intentional and forced choices around language, schedule, pronouns, and even technical terminology can make contributors from varying backgrounds feel welcome or unwelcome, or render the community inaccessible entirely. These too are political choices. Your post above is one of them.

                                                                                                                    There is, unfortunately, no such thing as a truly neutral stance on inclusion. Consider: you wish to take only the best developers, and yet your post has already discouraged good engineers from working on your project. Doubtless it has encouraged other engineers (who may be quite skilled!) with a similar political view to your own; those who believe, for instance, that current minority representation in tech is justified, representing the best engineers available, and that efforts to change those ratios are inherently discriminatory and unjust.

                                                                                                                    Policies have impact. Consider yours.

                                                                                                                    1. 7

                                                                                                                      I don’t know if that was your goal, but this is one of the best arguments for positive discrimination I’ve read. Thanks for posting it, and also thanks for noting that all decisions have some inherent politics whether we like it or not.

                                                                                                                      Unfortunately there is simply no solution: positive discrimination is opposed to meritocracy. Forced ratios are definitely an unethical tool, as they are a form of discrimination. However, this unethical tool brings us to a greater good, which is a final product that incorporates diversity on its design and accommodates more users, which is a desirable goal on itself, for the reasons you explained.

                                                                                                                      1. 4

                                                                                                                        color schemes determine whether color-blind people are able to quickly scan active vs inactive options and understand information hierarchy. Font sizes and contrast ratios can make the interface readable, difficult, or completely unusable for visually impaired people. The sizes of click targets, double-click timeouts, and drag thresholds

                                                                                                                        Let me see if I understand what you’re saying. Are you claiming that when color schemes, font sizes and drag thresholds are chosen that that is a political decision? I think that many people would find that quite a remarkable claim.

                                                                                                                        1. 7

                                                                                                                          It’s impossible to not be political. You can be “the status quo is great and I don’t want to discuss it”, but that’s political. The open source “movement” started off political - with a strong point of view on how software economics should be changed. In particular, if you say a CoC that bans people from being abusive is unacceptable, you are making a political statement and a moral statement.

                                                                                                                          1. 3

                                                                                                                            It’s impossible to not be political

                                                                                                                            Could I ask you to clarify in what sense you are using the word “political”?

                                                                                                                            Merriam-Webster (for example) suggests several different meanings that capture ranges of activity of quite different sizes. For example, I’m sure it’s possible to act in a way which does not impinge upon “the art or science of government” but perhaps every (public) action impinges upon “the total complex of relations between people living in society”.

                                                                                                                            In what sense did you use that term?

                                                                                                                            1. 8

                                                                                                                              Let’s start off with a note about honesty. FRIGN begins by telling us “We do not believe in equal status (i.e. e.g. forcibly obtaining a 50/50 gender ratio)” as if someone was proposing the use of force to produce a 50/50 gender ratio - and we all know that wasn’t proposed by anyone. There’s no way to discuss this properly if people are going to raise false issues like that. What comment’s like FRIGN’s indicate is an unwillingness to have an open and honest conversation. The same bogus rhetoric is at the heart of Damore’s memo: he claims to be in favor of equal rights and just against mythical demand for 50/50 gender equality so that he can oppose obviously ineffective affirmative action programs at Google where 80% of technical staff are male (Damore’s misappropriation of science is similarly based on an objection to a position that nobody ever argued.).

                                                                                                                              The next point is that some people are objecting that a CoC and a minority outreach program are “political”. That’s true, but it involves the use of the more general meaning of “political” which the Collins dictionary provides as “the complex or aggregate of relationships of people in society, esp those relationships involving authority or power”. If we are using that definition, of course a CoC and a minority outreach program are political, but opposition to a CoC and a minority outreach program fits the definition as well. If you have an opinion one way or another, your opinion is political. You can’t sensibly use this wide definition of political to label the effort to adopt a CoC and to recruit more minorities and then turn around and claim your opposition to those is somehow not political. So that’s what I mean by “it is impossible to not be political”. The question is a political question and those who try to claim the high ground of being objective, disinterested, non-political for their side of the question are not being straightforward (perhaps it’s just that they are not being straightforward with themselves).

                                                                                                                              1. 3

                                                                                                                                I agree that a CoC, a minority outreach program, and opposition to a CoC all impinge upon “the complex or aggregate of relationships of people in society, esp those relationships involving authority or power”.

                                                                                                                                Would you also agree that there is a popular ideological political movement in favour of CoCs (some combination of the feminist, civil rights and social justice movements)? Perhaps there is also a popular ideological movement against CoCs (some combination of MRAs and the alt right). Are you also claiming that if one claims a “neutral” stance on CoCs one is de facto supporting one of these ideologies?

                                                                                                                                1. 5

                                                                                                                                  I’m not sure it is possible to have a neutral stance. In fact, I doubt it.

                                                                                                                                  1. 1

                                                                                                                                    Interesting! Do you also doubt it is possible to take any action that is neutral with regard to a political ideology?

                                                                                                                                    1. 4

                                                                                                                                      You are introducing something different. I don’t think you have to line up with one “side” or another, but you can’t avoid being a participant.

                                                                                                                                      1. 1

                                                                                                                                        You said “It’s impossible to not be political” so I’m trying to understand what you mean by that. So far I’m not clear whether you think every action is political. I’d appreciate it if you’d clarify your position.

                                                                                                                                        1. 5

                                                                                                                                          I’m making a very concrete assertion, which I sense does not fit into your schema. My assertion is that there is no neutrality on workplace equality and inclusion for anyone involved in the workplace. Anyone who, for example, participates in an open source development effort has a position on whether efforts should be made to make it more inclusive even if that position is “this is not important enough for me to express an opinion.”

                                                                                                                                          1. 1

                                                                                                                                            Thank you for clarifying. When you originally said “It’s impossible to not be political” I got the wrong impression.

                                                                                                                                            Do you also hold the same point of view when it comes to roughly comparable statements in other spheres? For example ‘Anyone who eats has a position on vegetarianism even if that position is “this is not important enough for me to express an opinion.”’?

                                                                                                                        2. 1

                                                                                                                          You’ve been quoted by LWN: https://lwn.net/Articles/753709/

                                                                                                                        3. 11

                                                                                                                          AKA shut up and hack? :)

                                                                                                                          1. 2

                                                                                                                            The suckless development process has no non-technical discussions?

                                                                                                                            How are the best developers identified?

                                                                                                                            1. 8

                                                                                                                              just curious, why would you need to identify the best developers? Wouldn’t the quality of their code speak for that?

                                                                                                                              1. 5

                                                                                                                                I also fail to see what the reasoning is. Just send your code, get the non technical discussions out.

                                                                                                                                1. 1

                                                                                                                                  Apparently, quoting @FRIGN from above, “to make your software suck less.”

                                                                                                                                2. 8

                                                                                                                                  How are the best developers identified?

                                                                                                                                  I think this is a totally reasonable question, and one I’d like to see the answer too–if for no other reason than it might help those of us on other projects find more objective metrics to help track progress with.

                                                                                                                                  Do you all at suckless use something like:

                                                                                                                                  • defect rate
                                                                                                                                  • lines of code/feature shipped
                                                                                                                                  • execution time
                                                                                                                                  • space in memory, space in storage

                                                                                                                                  Like, what metrics do you use?

                                                                                                                                  1. 8

                                                                                                                                    You know, suckless is not a big company and the metrics that can be applied are more of a heuristic. A good developer is somebody who e.g. supplies a patch with a bug report, provides feedback to commits, makes contributions to the projects, thinks his commits through and doesn’t break stuff too often and does not personally identify with their code (i.e. is not butthurt when it’s not merged).

                                                                                                                                    What needs to be stressed here is that the metric “lines of code” is completely off. There are horrible programmers who spit out lots of code and excellent ones who over time drop more lines than they add. Especially the latter group is very present among us and thus the LOC-metric will only give false results. Same with execution time, you find that when not enough time is spent on a problem you end up solving it wrong, in the worst case having to start all over.

                                                                                                                              2. 8

                                                                                                                                By being very diverse and doing fackelmärsche of course. https://suckless.org/conferences/2017/

                                                                                                                                1. 6

                                                                                                                                  @FRIGN What’s the purpose of this “torchlight hike” in the context of producing code that sucks less? Don’t you see that the activities you choose to have during your conferences are a cultural stance, and because of that, can be perceived as exclusive by programmers that don’t recognize themselves in these activities?

                                                                                                                                  1. 2

                                                                                                                                    I get your point, but must honestly say that your argument sadly aligns with the ever-excluding and self-segregating destructful nature of cultural marxism. By eating food together at the conferences, do we exclude anorexics that might otherwise be willing to attend such a conference? I don’t drink any alcohol and never have. Still, it was not a problem when we went to a local Braukeller and some people drank alcohol and others like myself didn’t.

                                                                                                                                    The fundamental point I think is that one can never fully and analytically claim that a certain process is completely unaffected by something else. If we dive down into these details we would then move on and say that the different choice of clothings, hairstyle, means of travel and means of accomodation all affect the coding process at suckless. This can be taken further and further with no limit, as we all know about the butterfly effect. At some point it is just not measurable any more.

                                                                                                                                    If you ask me, this is a gross overstretching of what I said. There are quite a lot of people who do not attend the conferences but still work together with us on projects during that time. What really matters is that we e.g. do not ignore patches from these people or give them less relevance than those of others. To pick the example up: The torchlight hike did not affect any coding decision in a direct way, but it really bonded the team further together and was a very nice memory of this conference that I and the others are very fond of from what I’ve heard. On top of that, during the hike we were able to philosophize about some new projects of which some have become a reality. The net-gain of this event thus was positive.

                                                                                                                                    In classical philosophy, there are two main trains of thought when it comes to evaluating actions: Deontology and Teleology. Deontology measures the action itself and its ethical value, completely ignoring the higher goal in the process. Teleology is the opposite, evaluating actions only by their means to reach a goal, completely ignoring the value of the action itself. The best approach obviously should be inbetween. However, there is a much more important lesson that can be taken from here: When evaluating a decision, one needs to realize what they are measuring and what is unimportant for a decision. What I meant is that to reach the goal of software perfection, the gender and other factors of the submitters do not matter. So even though we here at suckless have a goal, we are not teleologists, as we just ignore the factors that do not matter for coding.

                                                                                                                                    It is an ethical question which norms you apply to a decision.

                                                                                                                                    If we look at organizations like Outreachy, one might be mistaken to think that they are deontologists, striving to improve processes. However, after closer inspection it becomes clear that this is not the case and they are actually working towards a certain goal, increasing the number of trans and minority people in such communities. No matter how you think about this goal, it makes one thing clear: When you are working towards such a goal and also do not ignore irrelevant factors in your norms (and they in fact do by not ignoring e.g. race and gender), you quickly end up discriminating against people.

                                                                                                                                    I hope this clears this up a bit, but as a short sentence, what can be taken from here is: When discussing ethical matters, it’s always important to make clear which norms are applied.

                                                                                                                                    1. 15


                                                                                                                                      I’m not going to wade into anything else on this, but I’d like to just take a second and let you know that, while you may not mean it in this way the phrase “cultural marxism” is very, very often used as a stand in for “jews”. Some links for the record:


                                                                                                                                      https://newrepublic.com/article/144317/trumps-racism-myth-cultural-marxism https://www.smh.com.au/world/cultural-marxism--the-ultimate-postfactual-dog-whistle-20171102-gzd7lq.html

                                                                                                                                      1. 12

                                                                                                                                        wow, uh, kind of a weird red flag that pointing this out is getting seriously downvoted. I picked these links pretty quickly, and anybody who comes behind and reads this and wonders how serious this is, do yourself a favor and image search and see how many memes have the star of david, greedy merchant, world strangling octopus or any of a number of openly anti-semitic imagery. Its not hidden, its not coy. If you’re tossing “cultural marxism” around you’re either willfully ignoring this or blatantly playing along. Its not a thing in the world. There are no leftists (at all) who call themselves “cultural marxists”, and in fact there is a sizeable faction of marxists who are openly disdainful of any marxism that eschews political struggle. The new republic article linked above goes into this, Perry Andersons “Considerations on Western Marxism”, a well known, well regarded text across a number of marxist subsects, is explicitly based on this. Anyway, enjoy contributing to a climate of increasing hostility toward jews. good stuff.

                                                                                                                                        edit: have some fun with this https://www.google.com/search?q=cultural+marxism&client=firefox-b&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjz2tWrhvnaAhUJ7YMKHVgcCccQ_AUIDCgD&biw=1247&bih=510#imgrc=_

                                                                                                                                        1. 3

                                                                                                                                          The term ‘Cultural Marxism’ describes very well what it is, and not all leftists are cultural marxists. The classical theory of marxism, roughly spoken, is to think of society as being split in two camps, the Proletariat and the Bourgeoisie, eternally involved in a struggle, where the former is discriminated against and oppresed by the latter.

                                                                                                                                          Cultural Marxism applies these ideas to society. In the Frankfurt School it was called ‘Critical Theory’, calling people out to question everything that was deemed a cultural norm. What is essentially lead to was to find oppressors and oppressed, and we reached the point where e.g. the patriarchy oppressed against women, white people against minorities, christians against muslims and other religions and so forth. You get the idea. Before you go again rallying about how I target jews or something please take a note that up to this point in this comment, I have just described what cultural marxism is and have not evaluated or criticized it in any way, because this here is the wrong platform for that.

                                                                                                                                          What you should keep in mind is that the nature of cultural marxism is to never be in a stable position. There will always be the hunt for the next oppressor and oppressed, which in the long run will destroy this entire movement from the inside. It was a friendly advice from my side to you not to endulge in this separatory logic, but of course I understand your reasoning to the fullest.

                                                                                                                                          Just as a side note: I did not see you getting ‘seriously’ downvoted. What do you mean?

                                                                                                                                          1. 8

                                                                                                                                            Generally people who use “cultural marxism” as a pejorative are sloganeering. The idea of an “eternal struggle” is completely foreign to any kind of marxism which is based on a theory that classes come out of the historical process and disappear due the historical process. Marxism claims that the proletariat and bourgeosie are temporary divisions that arise from a certain type of economic organization. Whatever one thinks of that idea, your characterization of Marxism is like describing baseball as a game involving pucks and ice. Your summary of “cultural marxism” is even worse. Maybe take a class or read a decent book.

                                                                                                                                            1. 10

                                                                                                                                              Did you actually read any of the links I posted? Specifically the New Republic and SPLC links? I don’t know how else to say this and you pretty much side stepped what I said the first time so I’ll try to reiterate it: There is no such thing as “Cultural Marxism”. At all. Its not a descriptive category that any marxist actually self applies or applies to other marxists. I’m fully aware of the Frankfurt School, Adorno, Horkheimer, etc. I’ve read some of them and many, many of their contemporaries from Germany, people like Karl Mannheim. I read marxist publications everyday, from here in the states and from Europe. I’m a member of an explicitly marxist political party here in the states. I can’t emphasize this enough, “cultural marxism” isn’t real and is roughly on par with “FEMA camps”, “HARRP rays” and shape shifting lizard jews, meaning; its a far far right wing paranoid fantasy used to wall off people from other people and an actual understanding of the material conditions of their world. I also didn’t say, specifically in fact pointing out that I wasn’t saying this, that you were “targeting jews”. That being said, if you use a phrase that has its origins in anti-semitic polemics, is used explicitly and over-whelmingly by anti-semites, than that is on you. (Did you take a look at the linked image search? Does that sort of thing not give you pause?) To say that you “just described what cultural marxism is” is also inaccurate, you absolutely used it in a descriptive way

                                                                                                                                              I get your point, but must honestly say that your argument sadly aligns with the ever-excluding and self->segregating destructful nature of cultural marxism.

                                                                                                                                              White supremacist organizing is experiencing an enormous upsurge, not only here in the states but in Europe as well. From Le Pen to AfD to SVO in Austria and on and on. These people are not interested in polite conversation and they’re not using “cultural marxism” as a category to illuminate political opponents, its meant to denigrate and isolate, ironically given thats exactly what Neo Nazis and white supremacists here in the states accuse left wingers and “SJWs” of doing.

                                                                                                                                              I appreciate that you’re discussing this peacefully but I’m going to bow out of this thread unless you’re interested enough to take some time and read the links

                                                                                                                                              FWIW these also dismantle the trope and point out pretty much exactly what I’m saying around anti-semitism: https://www.vice.com/en_us/article/78mnny/unwrapping-the-conspiracy-theory-that-drives-the-alt-right https://www.theguardian.com/commentisfree/2016/feb/22/chris-uhlmann-should-mind-his-language-on-cultural-marxism

                                                                                                                                              1. 3

                                                                                                                                                I took some more time to read it up and from what I could see, I found that indeed cultural marxism has become more of a political slogan rather than a normal theoretical term in the USA.

                                                                                                                                                Here in Germany the term “Kulturmarxismus” is much less politically charged from what I can see and thus I was surprised to get this response after I just had “translated” this term into English. It might be a lesson to first get some background on how this might be perceived internationally, however, it is a gigantic task for every term that might come around to you.

                                                                                                                                                So to reiterate my question, what term could be better used instead? :)

                                                                                                                                                1. 2

                                                                                                                                                  interesting that it has a different grounding/connotation in Germany, but then again I’m not surprised since thats where its supposed to have originated from. I’ll reread your other posts and come up with a response thats fair. Thanks for taking the time to read those links.

                                                                                                                                              2. 2

                                                                                                                                                It’s uncommon to find such a well-put explanation; thanks for that.

                                                                                                                                                There will always be the hunt for the next oppressor and oppressed, which in the long run will destroy this entire movement from the inside.

                                                                                                                                                If the movement runs out of good targets (and falls apart because they can’t agree on new ones), wouldn’t that imply that it will self-destruct only after it succeeds in its goals? That doesn’t sound like a bad thing.

                                                                                                                                                1. 2

                                                                                                                                                  I’m glad you liked my explanation. :)

                                                                                                                                                  That is a very interesting idea, thanks for bringing this thought up! It’s a matter dependent on many different factors, I suppose. It might fall apart due to not being able to agree on new targets or when everybody has become a target, but it is a very theoretical question which one of these outcomes applies here.

                                                                                                                                            2. 5

                                                                                                                                              It’s not my fault that some idiots don’t understand this term or it’s critical analysis. Cultural marxism, as the term implies, is the classical theory of marxism applied to culture. It has nothing to do with jews directly, it’s just an idea. If you know any better term to describe it, please let me know.

                                                                                                                                              Anyway, in the philosophical realms it’s known as ‘Critical Theory’, which originated in the Frankfurt School. However, nobody knows this term.

                                                                                                                                              Unless a better term is found, I disregard your argument and won’t accept your attempt to limit language of perfectly acceptable words to describe an idea. At the end of the day, terminology must be found that adequately describes what a certain idea is, and I see no reason why this should be wrong.

                                                                                                                                              Regarding the torch hike: Yes, marching with torches was abused by the NSDAP as a means of political rallying. However, at least in Germany, it is a much older and deeper-reaching tradition that dates back hundreds of years.

                                                                                                                                              1. 2

                                                                                                                                                You have amply demonstrated that you don’t know anything about the topic. You could start with the decent Wikipedia article. https://en.wikipedia.org/wiki/Frankfurt_School

                                                                                                                                    2. 17

                                                                                                                                      I’m not going to remove this because you’re making a public statement for suckless, but please don’t characterize positions you disagree with as madness. That kind of hyperbole generally just leads to unproductive fights.

                                                                                                                                      1. 9

                                                                                                                                        Please don’t remove anything unless it’s particularly vulgar…

                                                                                                                                        1. 0

                                                                                                                                          Removing differing viewpoints? It is precisely this kind of behavior that maddens people who complain about SJW, who (the SJW) seem unable to take any discussion beyond calling their opponent’s position “evil”, “alt-right”, “neo-nazi”, or, if they are exceptionally well-spoken, “mad”.

                                                                                                                                          1. 15

                                                                                                                                            No, removing abuse and hyperbole that acts as flamebait regardless of the political opinions expressed. So far I’ve removed one post and hope not to remove more.

                                                                                                                                            1. 3

                                                                                                                                              It’s hard for me to see a reason to remove things when we have the voting system in place, neither are perfect but one is at your sole discretion whereas the other is the aggregate opinion of the users.

                                                                                                                                              1. 22

                                                                                                                                                Voting isn’t a replacement of moderation. It helps highlight and reward good comments and it can punish bad comments, but it’s not sufficient for running a community. I’m trying to head off places where people give up on argument and just try to hurt or tar the people they disagree with because it doesn’t lead to a good community. Lobsters is a very good place for discussing computing and I haven’t seen that in communities this size with hands-off moderation (but I’d love counter-examples to learn from!) From a quick query, we’ve had comments from 727 unique users in the last 30 days and there’s around 15k unique IPs in the logs per weekday, so people are constantly interacting with the others who don’t know their background, don’t share history, can’t recognize in-jokes, simply don’t have reason to trust when messages are ambiguous, let alone provocative. Friendly teasing like “ah yeah, you would think that” or “lol php sucks” that’s rewarding bonding in a small, familiar group hurts in a big one because even if the recipient gets the joke and laughs along or brushes it off as harmless, it’s read by thousands of people who don’t or can’t.

                                                                                                                                                1. 2

                                                                                                                                                  Lobsters is a very good place for discussing computing and I haven’t seen that in communities this size with hands-off moderation

                                                                                                                                                  I support your position on sub-topic but even my Trial you linked to shows a bit otherwise on just this point. This site has more flexible, hands-off moderation than many I’ve seen with this much political dispute. Even in that link, we saw an amount of honest, civility, and compromise I don’t usually see. There’s been quite a bit better results in this thread than usual elsewhere. There seems to be enough community closeness despite our size that people are recognizing each others positions a bit. Instead of comments, you can actually see it by what’s not said more since it’s prior ground we’ve covered. The others are learning as discussion furthers. Then, there’s the stuff we don’t want which seems to be basically what those individuals are intending in a way that has nothing to do with site’s size.

                                                                                                                                                  So, I support you getting rid of just pure abuse, trolling, sockpuppeting, etc. I don’t think we’ve hit the full weaknesses and limited vision of large sites yet despite our increase in comments and views. We’re still doing a lot better than average. We’re still doing it with minimal intervention on things like politics relative to what I’ve seen elsewhere. I think we can keep at current moderation strategy for now because of that. For now.

                                                                                                                                                  Just wanted to say that in the middle of all this.

                                                                                                                                                  1. 0

                                                                                                                                                    Voting isn’t a replacement of moderation. It helps highlight and reward good comments and it can punish bad comments, but it’s not sufficient for running a community.

                                                                                                                                                    I’m not sure if I see why it’s not a good replacement. To me, I see voting as distributed moderation and the “real” moderation is automatically hiding (not removing) comments when they fall below a threshold.

                                                                                                                                                    I’m trying to head off places where people give up on argument and just try to hurt or tar the people they disagree with because it doesn’t lead to a good community.

                                                                                                                                                    I think this method relies on an accurate crystal ball where you can foresee people’s actions and to an extent, the reactions of the people reading the comments.

                                                                                                                                                    I’d have to question what you mean by “a good community”, it seems like it’s just a place where everyone agrees with what you agree with and those that disagree aren’t heard because it risks offending those that do agree.

                                                                                                                                                    I think the best discussions on here are because we have many people with wide and varied opinions and backgrounds. The good comes from understanding what someone else is saying, not excluding them from the discussion. The only places I see that warranted is where someone has said something purposely and undeniably vile.

                                                                                                                                                    1. 9

                                                                                                                                                      The automatic hiding of low-scoring comments is also a “sole discretion” thing; jcs added it and I tweaked it a few months ago. The codebase enforces a lot of one moderator’s ideas of what’s good for a community in a hands-off way and the desire to do that motivated its creation.

                                                                                                                                                      I strongly agree that a community where everyone agrees with the moderator would be bad one, even if I am that moderator. It’s tremendously rewarding to understand why other people see things differently, if for no other reason than the selfish reason that one can’t correct learn or correct mistakes if one never sees things one doesn’t already agree with.

                                                                                                                                                      I think the crystal ball for foreseeing problems is experience, from many years of reading and participating in communities as they thrive or fail. I think it’s possible to recognize and intervene earlier than the really vile stuff because I’ve seen it work and I’ve seen its absence fail. I keep asking for examples of excellent large communities without active moderators because I haven’t seen those, and after a couple decades and a few hundred communities I see the anthropic principle at work: they don’t exist because they self-destruct, sink into constant vileness, or add moderation. At best they have maintain with signal-to-noise ratios far below that of Lobsters where the thoughtful commentary is crowded out by trolling, running jokes, ignorance, and plan low-quality comments because it doesn’t seem worth anyone’s while to care when posting.

                                                                                                                                                      But moderation is not a panacea in and of itself. Without good experience, judgment, and temper a bad moderator swiftly destroys a community, and this is a very common way communities fail. If it helps any, the author of the comment I removed agrees that it wasn’t done to suppress their opinion.

                                                                                                                                                      1. 1

                                                                                                                                                        The benefit I see from moderation being part of the codebase is that it’s public, predictable and repeatable (it terms of reliability). When you take moderation decisions into your own discretion many of these virtues are lost.

                                                                                                                                                        As for experience, I think that’s tricky because it can easily lead you to making the same mistake twice. It’s also made of your personal experiences and you’re using that to curate the discussion of other people, I would caution that it’s another method of controlling dialog (perhaps subconsciously) to what you find acceptable, not necessarily what’s best for everyone.

                                                                                                                                                        1. 3

                                                                                                                                                          The benefit I see from moderation being part of the codebase is that it’s public, predictable and repeatable (it terms of reliability). When you take moderation decisions into your own discretion many of these virtues are lost.

                                                                                                                                                          Most of them go into the Moderation Log. I’ve been watching it since the jcs days since it’s what folks are supposed to do in a transparent, accountable system. Gotta put effort in. I haven’t seen much of anything that bothered me. The bans and deletes I’ve been able to follow @pushcx doing were trolling, alleged sockpuppeting, and vicious flamewaring. Some I couldn’t see where I’d rather the resource go off the front page rather getting deleted so someone looking at logs could see it for whatever it was. Nonetheless, his actions in the thread about me, the general admining, and what I’ve seen in moderation have been mostly good. A few really good like highlighting the best examples of good character on the site. I think he’s the only one I’ve seen do that on a forum in a while.

                                                                                                                                                          You have little to worry about with him in my opinion at the moment. Do keep an eye on the comments and log if you’re concerned. Scrape them into version storage if concerned about deletions. What goes on here is pretty public. Relax or worry as much as you want. I’m more relaxed than worried. :)

                                                                                                                                                          1. 3

                                                                                                                                                            Yeah, I agree on the pitfalls of experience. As SeanTAllen noted in a separate branch of this thread a minute ago, there’s “but you didn’t say” and other wiggle room; I think that’s where automatic moderation falls down and human judgment is required. Voting has its own downsides like fads, groupthink, using them to disagree (which is all over this thread), in-jokes, a drifting definition of topicality, all the parallels to the behaviors of political rhetoric, etc. Lobsters has never been voting only and I don’t see a compelling reason to change that. jcs’s involvement in the site was steadily declining so I’m certainly more actively moderating, but I don’t see that as a change in character. I guess what it comes down to is that I agree with you about what successful communities do and don’t look like, but I haven’t seen one that works on the model you’ve outlined and I don’t see that kind of fundamental change as a risk worth taking.

                                                                                                                                                2. 2

                                                                                                                                                  So FRIGN writes to oppose “SWJ madness”, and you chime in to complain that “SWJ” calls opponents “mad”. Are you calling FRIGN “SWJ” or what? It’s kind of hard to discern your point in that cloud of grievance.

                                                                                                                                                  1. 2

                                                                                                                                                    “SJW” for “social justice warrior.”

                                                                                                                                                    @COCK is sarcastically non-replying because you typo’ed.

                                                                                                                                                    1. 2

                                                                                                                                                      Not exactly, I was sarcastically non-replying because I assumed he was intentionally misunderstanding me. I assumed this because I didn’t see any ambiguity in my answer. On later inspection I noticed the ambiguity so I gave an actual reply:


                                                                                                                                                      1. 1

                                                                                                                                                        The interesting thing is how people agreeing with Mr. cock pile on the insults against the people who they complain are insulting them by forcing them to sign on to codes of conduct which prohibit insults. It’s almost as if there was a good reason for those codes.

                                                                                                                                                        1. 1

                                                                                                                                                          I doubt the irony is lost on anyone supporting a CoC.

                                                                                                                                                      2. -1

                                                                                                                                                        Yes, I’m calling FRIGN a “SWJ”.

                                                                                                                                                      3. 1

                                                                                                                                                        The only comment that was removed was against FRIGN point of view. Nobody is removing differing point of view, just enforcing civil discussion.

                                                                                                                                                    2. 5

                                                                                                                                                      It’s responses like yours that really make the case for codes of conduct.

                                                                                                                                                      1. 2

                                                                                                                                                        Are you speaking for the group or is that your own opinion? Knowing that the group aligns itself with that position would certainly make me not interested in working with it or contributing.

                                                                                                                                                        1. 7

                                                                                                                                                          To be fair, suckless is not well-organised enough to be a group that can have a single opinion to be spoken for.

                                                                                                                                                          That said, FRIGN is a prominent contributor and I from what I’ve seen most contributors are heavily on the side of “the code will speak for itself”.

                                                                                                                                                      1. 1

                                                                                                                                                        I have what feels like a really stupid question:

                                                                                                                                                        A careful investigation of the raw history reveals that just prior to the read of 4, process 98 attempted to write 4, and received a failure code :unavailable:

                                                                                                                                                        Isn’t it always legal for any DB client to claim that an operation failed when it actually succeeded? e.g. the op may have succeeded but the acknowledgement from the DB server got lost. I feel like I missed something obvious.

                                                                                                                                                        1. 4

                                                                                                                                                          Yes, this is a subtle question! Two-generals implies that we cannot determine whether or not a message was received in a finite number of messages. This means that any request-response pattern has three possible outcomes: a definite success, a definite failure, or an indeterminate result (e.g. crash, timeout, …) which could be either successful or not; we can’t say. Jepsen is careful to treat each of these cases appropriately. In this particular case, the database returned a definite failure error, rather than an indeterminate one. When databases say something definitely didn’t happen, we hold them to it.

                                                                                                                                                          1. 1

                                                                                                                                                            Thank you! ❤

                                                                                                                                                            So if it had returned an error code for which the documentation said “your update might have been applied despite this error. We can’t tell for sure” then you would have made the linearizability testing treat that as indeterminate rather than success or failure, and that history would have been considered legal?

                                                                                                                                                            1. 2

                                                                                                                                                              Yes, exactly. By default, Jepsen treats all errors as indeterminate ones, and we allow both possibilities. You have to explicitly tell Jepsen that a certain error is a definite failure, and then it won’t consider that operation as one that could have happened, when it goes to check for correctness. :)

                                                                                                                                                              1. 1

                                                                                                                                                                Thank you. ♥

                                                                                                                                                        1. 4

                                                                                                                                                          Great writeup as always @aphyr!

                                                                                                                                                          Out of curiosity, what sort of pricing do you give for this sort of work, and is it per-time-period or per-product or per-defect found or what?

                                                                                                                                                          1. 9

                                                                                                                                                            Thank you.

                                                                                                                                                            I usually charge for dedicated weeks of time, and we keep going as long as the client feels the work is fruitful, but it varies. Some work I try to do for free, or with a pre-arranged rate in installments.

                                                                                                                                                            1. 1

                                                                                                                                                              Have you done CockroachDB yet?

                                                                                                                                                              1. 1

                                                                                                                                                                Yep! Review is here: https://jepsen.io/analyses/cockroachdb-beta-20160829 (full disclosure: I work at cockroach)

                                                                                                                                                          1. 31

                                                                                                                                                            Author here. To be clear, all the stories in this series are jokes, not endorsements. Engineering interviews are a complex problem and I won’t begin to discuss them here, except to say that there are people whose entire lives involve studying and improving teams of humans, they’re called organizational psychologists, and it might be worth hiring some.

                                                                                                                                                            Since folks have expressed incredulity that these techniques are at all involved in modern programming jobs, I should note that I have had to implement trees, sorting algorithms, and parsers in my professional code. For instance, I wrote the clj-antlr parser bindings to support Riemann’s query parser, which is chock full of tree transforms, on account of it being a compiler for search predicates. Knossos includes a hand-rolled binary search over… what are essentially packed and bitmasked structs, masquerading as a JVM int array. There’s also some graph neighborhood stuff in there that allows us to memoize arbitrary function calls over objects into pointer-chasing through densely packed int arrays. Gretchen implements a number of simplification algorithms over trees of boolean expressions, as well as Tseitin-expansion to reduce predicates to conjunctive normal form, which are required to turn transaction dependency graphs into constraint- and SAT-solver-friendly expressions. This is, like, the third or fourth time I’ve tackled this problem.

                                                                                                                                                            I don’t… think I’ve done a regex algorithm yet, but my former coworker Zach Tellman has done quite a few DFA and NFAs in his time; Automat comes to mind. I’m sure it’s only a matter of time before I wind up doing it too.

                                                                                                                                                            My experience is, of course, not representative. :)

                                                                                                                                                            1. 1

                                                                                                                                                              Why is it not representative? Seems like problems any experienced person may encounter in their travels. Enjoyed the story BTW.

                                                                                                                                                              1. 3

                                                                                                                                                                That’s the hope, but I suspect not everyone gets the chance to venture out into the interesting waters of comp sci.

                                                                                                                                                                1. 3

                                                                                                                                                                  I’d guess because the median programmer is writing business code and never writes library code in their entire career.

                                                                                                                                                                  1. 1

                                                                                                                                                                    Come on, that’s rock-star stuff.

                                                                                                                                                                    1. 1

                                                                                                                                                                      It’s true, it’s obviously great work. My point is that aphyr IS representative of experienced people. Experienced people … experience many variety of problems.

                                                                                                                                                                1. 18

                                                                                                                                                                  This is something that the rest of the team and I have been working on for more than a year. I think open source sustainability is going to be one of the big issues the tech community needs to face in the next ten years, in all communities, but particularly in niche ones like Clojure. Ruby Together has a really good model, so we copied it and applied it to the Clojure community (with some tweaks). Happy to answer any questions people have.

                                                                                                                                                                  1. 17

                                                                                                                                                                    Thank you for putting this together–all of you. I’m signing up Jepsen as a corporate sponsor right now.

                                                                                                                                                                  1. 1

                                                                                                                                                                    For testing byzantine faults, it’s important to keep results actionable. https://github.com/madthanu/alice does a nice job of introducing realistic filesystem semantics for crash testing. You may need to run it on the ubuntu version they recommend, as their patched strace is a little out of date. It’s a pretty simple tool to use other than that!

                                                                                                                                                                    1. 4

                                                                                                                                                                      Jepsen is a distributed systems test first and foremost, but yes, for single-node faults, tools like Alice are nice. I actually spent a week or so on a research project involving filesystem-level faults, but it didn’t product useful results within the time I had available.

                                                                                                                                                                      1. 2

                                                                                                                                                                        Recovery correctness has a ton of implications for distributed systems. It’s vital for leader election in broadcast-replicated systems with progress-related invariants, like what raft needs to enforce. I’ve also come across several popular (purportedly linearizable) distributed databases that will do things like start serving before their recovery process completes, returning stale reads just behind the in-progress recovery scan of the WAL. You’ll find gold if you look.

                                                                                                                                                                        1. 5

                                                                                                                                                                          Wow thank you, that sounds like a really interesting research opportunity!

                                                                                                                                                                    1. 4

                                                                                                                                                                      If you’d like to see an example of how these numbers play out in practice, take a look at Zach Tellman’s benchmarks for Bifurcan (https://github.com/lacuna/bifurcan/blob/master/doc/benchmarks.md), which compare Java’s ArrayList to Clojure’s vectors (which, like Scala’s vectors, are 32-way hash array mapped tries), and also Java’s HashMap and HashSet to Clojure’s hashmaps. You can also see how some of the decisions in his linear and persistent collections (List vs Linearlist, Map vs LinearMap, etc) affect performance.

                                                                                                                                                                      1. 3

                                                                                                                                                                        Out of curiosity, when you evaluated Knossos performance, were you testing using histories with crashed clients, or happy-path histories where client operations succeed relatively quickly? Knossos makes some choices which optimize for the former, and I think the P-compositionality paper focuses on the latter, but it’s been a few years since I’ve been down in those papers guts. May need to revisit those assumptions if they were slower for your workload.

                                                                                                                                                                        1. 3

                                                                                                                                                                          Hi aphyr!

                                                                                                                                                                          To make the comparison more fair to Knossos, I tested histories where you can’t take advantage of P-compositionality. (in short, P-compositionality is when a history is linearizabile iff all subhistories in a partitioned history are linearizable - e.g. with a map, you can partition by keys and check the subhistories independently, and that’s a lot faster)

                                                                                                                                                                          I used test data from Jepsen etcd tests: https://github.com/anishathalye/porcupine/blob/master/test_data/jepsen

                                                                                                                                                                          Here’s the quick-and-dirty benchmarking code I used: https://gist.github.com/anishathalye/a315a31d57cad6013f57d2eb262443f5 (basically, just timing knossos.core/linearizable-prefix).

                                                                                                                                                                          Even where Knossos is slow, e.g. etcd_002.log and etcd_099.log (timed out after > 2 days). Porcupine seems to do fine, taking hundreds of milliseconds on a single core to check the histories.

                                                                                                                                                                          Out of the ~100 tests, filtering for ones that Knossos finished in < 1 hour, we have a speedup of 1000x on Knossos’s fastest test (etcd_016.log) and a speedup of 20,000x on Knossos’s slowest test (etcd_040.log). And for the ones that timed out (because I didn’t want to run the tests for way too long), e.g. (etcd_099.log), Porcupine had a speedup of > 10^6.

                                                                                                                                                                          I haven’t had time to look into Knossos’s implementation in detail and figure out exactly where Porcupine’s speedups are coming from, but that would be cool to do at some point. Jepsen/Knossos are obviously a way more complete solution for testing distributed systems, and it would be cool to speed up the linearizability checking aspect.

                                                                                                                                                                          1. 2

                                                                                                                                                                            Ohhhhhh! Yeah, you’re using the original algorithm–that’s definitely slower. Try (knossos.linear/analysis model history) instead–that’s based on the JIT-linearization algorithm from Lowe’s paper, plus some additional optimizations–instead of performing union-find for compaction, we pre-compile the state space into a graph of pointer arrays, which turns the search into immutable pointer-chasing instead of running model code. There are… certain cases where the knossos.core algorithm is preferable (it offers better parallelism) but linear should be significantly faster. Still not good though; I’d like to sit down and figure out some alternative strategies.

                                                                                                                                                                            And yeah, as you note, we don’t do P-compositionality in Knossos–that’s handled by Jepsen, which performs the decomposition into independent keys for maps, sets, etc, then hands individual histories to Knossos. Would be nice to fold into Knossos later though!

                                                                                                                                                                            Last thing, if you wind up packaging this for distribution, I’d like to offer a hook in Jepsen so we can pass histories to it. If you’d like to define some sort of serialization format (JSON, tsv, EDN, protobufs, etc) for passing histories in and getting analysis results back, I can wrap that up in a Jepsen checker as an alternative strategy. :)

                                                                                                                                                                            1. 2

                                                                                                                                                                              Oops, I didn’t know that. I redid the benchmarking with (knossos.linear/analysis model history), running Knossos on 6 cores and Porcupine on 1 core.

                                                                                                                                                                              The benchmark results did change: Knossos completed every check significantly faster. On some tests, the new algorithm performed significantly better: e.g. on etcd_040.log, Porcupine has a speedup of 278x, as opposed to a speedup of 20,000x when comparing against the original algorithm (knossos.core/linearizable-prefix).

                                                                                                                                                                              Porcupine still ran faster on all tests, though; the following is a summary of the results (over the ~100 Jepsen etcd tests):

                                                                                                                                                                              Min speedup:    8.1x     on etcd_002.log
                                                                                                                                                                              Max speedup:    21,219x  on etcd_067.log
                                                                                                                                                                              Median speedup: 1000x

                                                                                                                                                                              Ooh, that sounds cool! I’ll probably end up packaging this for distribution in a couple weeks, and I’ll definitely reach out to you once I have an API for communicating with Porcupine.

                                                                                                                                                                        1. 3

                                                                                                                                                                          The motherboard is wonky and refuses to find half the disks on boot. You can crash the box by using certain USB ports. We have a complicated relationship.

                                                                                                                                                                          I’m curious about this. You can buy a stable, reliable computer with 48 cores and 128GB of ECC RAM completely off the shelf — Dell/HP/etc sell rackmount or tower servers with these configurations.

                                                                                                                                                                          Is using a really powerful but unreliable desktop computer a net productivity advantage relative to using a small reliable desktop + SSHing into a big reliable server? I appreciate that remote debugging is often not as nice as local debugging, but on the other hand remote debugging has some nice side benefits like the fact that the UI that you’re using doesn’t go unresponsive when the machine gets loaded.

                                                                                                                                                                          I can totally sympathise if it turns out that the root cause of this was just that Kyle just really really wanted a really overpowered computer out of sheer nerdlust.

                                                                                                                                                                          1. 8

                                                                                                                                                                            sheer nerdlust

                                                                                                                                                                            I know it’s a time-honored tradition to armchair-architect strangers’ technical decisions without regard for the privileges of context or experience, but fuck it, I’ll bite.

                                                                                                                                                                            I used to rebuild and rack servers for a living, and considered that here, but ultimately decided I wanted a workstation.

                                                                                                                                                                            It’s quieter; I didn’t feel like having a screaming banshee 2u sitting in my tiny SF bedroom. It means owning one computer instead of two, which is cheaper, takes up less space, and cuts down on my time spent doing stupid sysadmin stuff. It’s also way less of a pain in the ass to work with than the janky-ass combination of remote filesystems, SSH tunnels, rsync hacks, X forwarding, and Yourkit injection that I have to use with remote Jepsen clusters.

                                                                                                                                                                            1. 6

                                                                                                                                                                              Thank you for replying! I’m sorry if I came off as insulting. Edit: I apologise for insulting you. That was not my intention.

                                                                                                                                                                              It’s quieter

                                                                                                                                                                              I appreciate the noise issue. I’m used to 1U servers being awful for it because the fans are small so the noise is high-pitched, haven’t gotten my hands on 2U or up to see if they’re much quieter. I thought tower servers were supposed to be no worse than desktops in this regard? Since they’re not that different and can use similarly huge fans?

                                                                                                                                                                              janky-ass combination of remote filesystems, SSH tunnels, rsync hacks, X forwarding, and Yourkit injection

                                                                                                                                                                              Ouch. Good point, avoiding that mess is worth a lot of effort.

                                                                                                                                                                            2. 1

                                                                                                                                                                              My guess is that its some kind of whitebox. I’ve never had good lucky with them, and always some kind of jank. I replaced my whitebox server with an SFF business desktop and it’s been far better in stability.

                                                                                                                                                                            1. 14

                                                                                                                                                                              VoltDB doesn’t have a whole lot to sell to me, really. That said, if I understand the chain of events properly, VoltDB sponsored this Jepsen post (and all the research that went into it), took those findings and started fixing the problems unearthed. That’s an admirable commitment to both data safety and openness, and it means I’ll consider VoltDB preferably over competitors should I ever need that feature set.

                                                                                                                                                                              1. 28

                                                                                                                                                                                Yep, you’re understanding correctly. Like RethinkDB, VoltDB approached me for help in testing their systems, and funded the research. I found initial cases pretty quickly, deeper problems over the next month, and worked with their team for the next month or so to create more stringent tests and evaluate proposed fixes–VoltDB ported some of these test scenarios to their internal test suite, and is integrating Jepsen into their testing cycle now. That work culminated in the release of 6.4 last week. You can read more about how I handle sponsored research, and see the full set of bugs we uncovered on VoltDB’s issue tracker.