1. 6

    Among the items being explored by systemd-homed are JSON-based user records

    oh god no

    1.  

      You like the GECOS field?

      1. 9

        Surely we can agree there is design space worth exploring between “the first thing the Unix devs originally hacked together in ’69” and “arbitrarily-structured unschema-ed generic serialization format with poor human readability and writability which is difficult to manipulate with plain-text tools and has few affordances for streaming reads”.

        1.  

          How often are you editing /etc/passwd and friends “raw”? I just checked my VPS, there’s 35 lines, of which one is my user (that was added when I set up this instance). The rest have been added programmatically by apt and friends.

          When I actually did have to add stuff (on *BSD) I used vipw to prevent me from messing up a field.

          I suspect most files for user directories etc are generated programmatically (via puppet etc) anyway. Why does the file have to be easy to handle in a text editor?

          1.  

            Well, if it doesn’t need to be human editable, why use JSON? It’s purest madness.

            1.  

              Because, in contrast to a binary format, you can fix it with vi in case it is really necessary.

              JSON is not my favorite text-based data format, but switching to something that is more structured that is still human readable, but also machine-modifiable sounds like a great idea. If it really needs to be JSON, I can live with it.

              Anyway, this borders on bike shedding, we are discussing JSON vs. TOML vs. GECOS, rather than the main ideas proposed in the presentation: decoupling user information from /etc, making home directories portable, better per-user home directory encryption, etc.

              1.  

                It’s purest madness.

                I don’t agree, but I’m open for suggestions for other formats.

                The entire idea is so much more than just “JSON for config” - there’s all sorts of stuff to make using a Linux computer better for (mostly) laptop users.

                1.  

                  Yeah, I push back against JSON, not the systemd stuff in the large – it doesn’t cause me much heartache, because I only have one Linux computer in my life, and I hate hate hate hate traditional SysV or BSD init garbage.

        2.  

          I’m looking forward to the discussions of whether systemd represents a use of JSON for good, or evil.

          1. 6

            Evil, obviously. Fortunately (?), they’re not going to use json.org’s parser; they’ll implement their own, with a bunch of brand-new never-before-seen security vulnerabilities.

            1.  

              For good.

            2. 5

              heck, what’s going on with young programmers today? Is there anything wrong with plain text files?

              1. 9

                we like data structures without having to write our own ad-hoc parser

                1.  

                  Well, does “plain text” include UTF-8?

                  1.  

                    Yes, it does. But you do not need json to encode utf8 strings.

                    1.  

                      The JSON spec states that the payload should be in UTF-8.

                      Edit I see what you mean now. Thanks for clarifying and thanks to @tentacloids for pointing it out for me even clearer.

                      1.  

                        I think this is a misunderstanding: @coco is saying you can use utf8 without involving JSON, not that JSON works with other encodings.

                    2.  

                      With punycode, yes, yes it does

                      1.  

                        Yeah, I really like my delicious shrimp sandwich[1] to be rendered as “xn–rksmrgs-5wao1o”.

                        And that adds a dependency to a punycode parser to your simple plain text format.

                        [1] Räksmörgås

                        1.  

                          It potentially adds a dependency to a punycode parser to your regular pipeline; it can be ignored in cases where you’re not looking for UTF-8 characters in the body of text.

                          1.  

                            In the context of a file describing a home directory you certainly do need to accommodate Unicode - unless you want to forever restrict usernames and directories to only ASCII.

                            I’m not saying JSON is the ideal format, just that alternatives cannot be restricted to ASCII.

                        2.  

                          Why the heck would you want to use punycode for this? Most ascii-centric algorithms, like Linux path parsing, work fine on valid UTF-8. And the underlying storage system is 8-bit clean. As long as you define the encoding, it shouldn’t be a problem.

                    3. 6

                      I don’t see the problem? JSON is reasonably okay and widely implemented, which makes it a good choice for something that may be manipulated by multiple parties.

                      1. 5

                        Until jq becomes part of coreutils, newline-separated records have much better shell support. I haven’t looked into the kind of manipulation that will be done to these user records, though.

                        1.  

                          I’d assume/hope that’s mostly privileged to the system itself and maybe moderated through the daemon.

                          On coreutils, I have not enough insight and you are probably right, but on the other hand find the coreutils path becoming antique.

                        2.  

                          It doesn’t support comments. TOML or one of the JSON variants would be more appropriate.

                          1.  

                            There’s a json fork called “hjson” (https://hjson.org/), which adds comment feature for json (among other small things), and it can be used for configuration files, but I doubt that it would be used by the systemd team.

                            Also I think TOML is pretty nice; trivial use makes it similar to .ini files, which systemd already uses, but at the same time it supports more complicated config structure.

                            1.  

                              TBH I think that all systemd service files should be moved to the TOML as it is more standardised and IMHO cleaner than their custom format.

                          2.  

                            I think it’s a bit surprising that they would some existing format; I assumed that the systemd devs would invent their own format.

                        1. 1

                          I’m surprised systemd is not involved here. ;)

                          1. 3

                            Downvoted as off-topic for blatant axe-grinding.

                            1. 1

                              Thanks for explaining the downvote. The smiley at the end was meant as an indication to not take it that seriously.

                          1. 6

                            The Nightly version of Firefox, so-called because it is updated every night, is a testing and development release of the browser that sits between the Developer Edition and the public release.

                            Wrong in the first sentence. The nightly channel doesn’t sit between anything: it’s the most frequently-updated release channel. Beta and Developer Edition both sit between Nightly and Stable.

                            I don’t want to come across as too nitpicky, but it is kind of annoying that they get this kind of thing wrong. It’s not like it’s a secret.

                            1. 23

                              Oh dang another essay on empirical software engineering! I wonder if they read the same sources I did

                              Reads blog

                              You watched the conference talk “What We Know We Don’t Know”, by Hillel Wayne, who, also disturbed by software’s apparent lack of scientific foundation, found and read as many scholarly papers as he could find. His conclusions are grim.

                              I think I’m now officially internet famous. I feel like I crossed a threshold or something :D

                              So I’m not sure how much of this is frustration with ESE in general or with me in particular, but a lot of quotes are about my talk, and so I’m not sure if I should be defending myself? I’m gonna err on the side of defending myself, mostly because it’s an excuse to excitedly talk about why I’m so fascinated by empirical engineering.


                              One thing I want to open with. I’ve mentioned a couple of times on Lobsters that I’m working on a long term journalism project. I’m interviewing people who worked as “traditional” engineers, then switched to software, and what they see as the similarities and differences. I’ve learned a lot from this project, but one thing in particular stands out: we are not special. Almost everything we think is unique about software, from the rapid iteration to clients changing the requirements after we’ve released, happens all the time in other fields.

                              So, if we can’t empirically study software engineering, it would follow that we can’t empirically study any kind of engineering. If “you can’t study it” only applied to software, that would make software Special. And everything else people say about how software is Special turns out to be wrong, so I think it’s the case here.

                              I haven’t interviewed people outside of engineering, but I believe it goes even further: engineering isn’t special. If we can’t study engineers, then we can’t study lawyers or nurses or teachers or librarians. Human endeavor is incredibly complex, and every argument we can make about why studying software is impossible extends to any other job. I fundamentally reject that. I think we can usefully study people, and so we can usefully study software engineers.

                              Okay so now for individual points. There’s some jank here, because I didn’t edit this a whole lot and didn’t polish it at all.

                              You were disappointed with Accelerate: The Science of Lean Software and DevOps. You agreed with most of its prescriptions. It made liberal use of descriptive statistics.

                              Accelerate’s research is exclusively done by surveying people. This doesn’t mean it’s not empirical- as I say in the talk, qualitative information is really helpful. And one of my favorite examples of qualitative research, the Gamasutra Study on Crunch Mode, uses a similar method. But it’s far from being settled, and it bothers me that people use Accelerate as “scientifically proven!!!

                              1. Controlled experiments are typically nothing like professional programming environments […] So far as I know, no researcher has ever gathered treatment and control groups of ten five-developer teams each, put them to work M-F, 9-5, for even a single month, in order to realistically simulate the conditions of a stable, familiar team and codebase.

                              You’d be surprised. “Two comparisons of programming languages”, in “making software”, does this with nine teams (but only for one day). Some labs specialize in this, like SIMULA lab. Companies do internal investigations on this- Microsoft and IBM especially has a lot of great work in this style.

                              But regardless of that, controlled experiments aren’t supposed to be holistic. They test what we can, in a small context, to get solid data on a specific thing. Like VM Warmup Blows Hot and Cold: in a controlled environment, how consistent are VM benchmarks? Turns out, not very! This goes against all of our logic and intuition, and shows the power of controlled studies. Ultimately, though, controlled studies are a relatively small portion of the field, just as they’re a small portion of most social sciences.

                              For that matter, using students is great for studies on how students learn. There’s a ton of amazing research on what makes CS concepts easier to learn, and you have to use students for that.

                              1. The unpredictable dynamics of human decision-making obscure the effects of software practices in field data. […] This doesn’t hold for field data, because real-life software teams don’t adopt software practices in a random manner, independent from all other factors that might potentially affect outcomes.

                              This is true for every form of human undertaking, not just software. Can we study teachers? Can we study doctors and nurses? Their world is just as chaotic and dependent as ours is. Yet we have tons of research on how educators and healthcare professionals do their jobs, because we collectively agree that it’s important to understand those jobs better.

                              One technique we can use cross-correlating among many different studies on many different groups. Take the question “does Continuous Delivery help”. Okay, we see that companies that practice it have better outcomes, for whatever definiton of “outcomes” we’re using. Is that correlation or causation? Next we can look at “interventions” where a company moved to CD and see how it changed their outcomes. We can see what practices all of the companies share and what things they have different, to see what cluster of other explanations we have. We can examine companies where some teams use CD and some teams do not, and correlate their performance. We can look at what happens when people move between the different teams. We can look at companies that moved away from CD.

                              We’re not basing our worldview off a single study. We’re doing many of them, in many different contexts, to get different facets of what the answer might actually be. This isn’t easy! But it’s worth doing.

                              1. The outcomes that can be measured aren’t always the outcomes that matter. […] So in order to effectively inform practice, research needs to ask a slightly different, more sophisticated question – not e.g. “what is the effect software practice X has on ‘defect rate’”, but “what is the effect software practice X has on ‘defect rate per unit effort’”. While it might be feasible to ask this question in the controlled experiment setting, it is difficult or impossible to ask of field data.

                              Pretty much all studies take this as a given. When we study things like “defect rate”, we’re always studying it in the context of unit time or unit cost. Otherwise we’d obviously just use formal verification for everything. And it’s totally feasible to ask this of field data. In some cases, companies are willing to instrument themselves- see TSP or the NASA data sets. In other cases, the data is computable- see research on defect rates due to organizational structure and code churn. Finally, we can cross-correlate between different projects, as is often done with repo mining.

                              These are hard problems, certaintly. But lots of things are “hard problems”. It’s literally scientists’ jobs to figure out how to solve these problems. Just because we, as layfolk, can’t figure out how to solve these problems doesn’t they’re impossible to solve.

                              1. Software practices and the conditions which modify them are varied, which limits the generality and authority of any tested hypothesis

                              This is why we do a lot of different studies and test a lot of different hypothesis. Again, this is an accepted fact in empiricial research. We know it’s hard. We do it anyway.

                              But if you’re holding your breath for the day when empirical science will produce a comprehensive framework for software development – like it does for, say, medicine – you will die of hypoxia.

                              A better analogue is healthcare, the actual system of how we run hospitals and such. Thats in the same boat as software development: there’s a lot we don’t know, but we’re trying to learn more. The difference is that most people believe studying healthcare is important, but that studying software is not.

                              Is this cause for despair? If science-based software development is off the table, what remains? Is it really true as Hillel suggests, that in the absence of science “we just don’t know” anything, and we are doomed to an era of “charisma-driven development” where the loudest opinion wins, and where superstition, ideology, and dogmatism reign supreme?

                              The lack of empirical evidence for most things doesn’t mean we’re “doomed to charisma-driven development.” Rather it’s the opposite: I find the lack of evidence immensely freeing. When someone says “you are unprofessional if you don’t use TDD” or “Dynamic types are immoral”, I know, with scientific certainty, that they don’t actually know. They just believe it. And maybe it’s true! But if they want to be honest with themselves, they have to accept that doubt. Nobody has the secret knowledge. Nobody actually knows, and we all gotta be humble and honest about how little we know.

                              Of course not. Scientific knowledge is not the only kind of knowledge, and scientific arguments are not the only type of arguments. Disciplines like history and philosophy, for instance, seem to do rather well, despite seldom subjecting their hypotheses to statistical tests.

                              Of course science isn’t the only kind of knowledge! I just gave a talk at Deconstruct on the importance of studying software history. My favorite software book is Data and Reality, which is a philosophical investigation into the nature of information representation. My claim is that science is a very powerful form of knowledge that we as software folk not only neglect, but take pride in our neglecting. It’s like, yes, we don’t just have science, we have history and philosophy. But why not use all three?

                              Your decision to accept or reject the argument might be mistaken – you might overlook some major inconsistency, or your judgement might be skewed by your own personal biases, or you might be fooled by some clever rhetorical trick. But all in all, your judgement will be based in part on the objective merit of the argument

                              Of course we can do that. Most of our knowledge will be accumulated this way, and that’s fine. But I think it’s a mistake to be satisfied with that. For any argument in software, I can find two experts, giants in their fields, who have rigorous arguments and beautiful narratives… that contradict each other. Science is about admitting that we are going to make mistakes, that we’re going to naturally believe things that aren’t true, no matter how mentally rigorous we try to be. That’s what makes it so important and so valuable. It gives us a way to say “well you believe X and I believe not X, so which is it?”

                              Science – or at least a mysticized version of it – can be a threat to this sort of inquiry. Lazy thinkers and ideologues don’t use science merely as a tool for critical thinking and reasoned argument, but as a substitute. Science appears to offer easy answers. Code review works. Continuous delivery works. TDD probably doesn’t. Why bother sifting through your experiences and piecing together your own narrative about these matters, when you can just read studies – outsource the reasoning to the researchers? […] We can simply dismiss them as “anti-science” and compare them to anti-vaxxers. […] I witnessed it play out among industry leaders in my Twitter feed, the day after I started drafting this post.

                              I think I know what you’re referencing here, and if it’s what I think it is, yeah that got ugly fast.

                              Regardless of how Thought Leaders use science, my experience has been the opposite of this. Being empirical is the opposite of easy. If I wanted to not think, I’d say “LOGICALLY I’m right” or something. But I’m an idiot and want to be empirical, which means reading dozens of papers that are all maddeningly contradictory. It means going through papers agonizingly carefully because the entire thing might be invalidated by an offhand remark.[1] It means reading paper’s references, and the references’ references, and trawling for followup papers, and reading the followup paper’s other references. It means spending hours hunting down preprints and emailing authors because most of the good stuff is locked away by the academic paper hoarders.

                              Being empirical means being painfully aware of the cognitive dissonance in your head. I love TDD. I recommend it to beginners all the time. I think it makes me a better programmer. At the same time, I know the evidence for it is… iffy. I have to accept that something I believe is mostly unfounded, and yet I still believe in it. That’s not the easy way out, that’s for sure!

                              And even when the evidence is in your favor, the final claim is infuriatingly nuanced. Take code review! “Code Review works”. By works, I mean “in most controlled studies and field studies, code review finds a large portion of the extant bugs in reviewed code in a reasonable timeframe. But most of the comments in code review are not bug-finding, but code quality things, about 3 code improvements per 1 bug usually. Certain things make CR better, and certain things make it a lot worse, and developers often complain that most of the code review comments are nitpicks. Often CRs are assigned to people who don’t actually know that area of the codebase well, which is a waste of time for everyone. There’s a limit to how much people can CR at a time, meaning it can easily become a bottleneck if you opt for 100% review coverage.”

                              That’s a way more nuanced claim than just “code review works!” And it’s way, way more nuanced than about 99% of the Code Review takes I see online that don’t talk about the evidence. Empiricism means being more diligent and putting in more work to understand, not less.


                              So one last thought to close this out. Studying software is hard. People bring up how expensive it is. And it is expensive, just as it’s expensive to study people in general. But here’s the thing. We are one of the richest industries in the history of the world. Apple’s revenue last year was a quarter trillion dollars. That’s not something we should leave to folklore and feelings. We’re worth studying.

                              [1]: I recently read one paper that looked solid and had some really good results… and one sentence in the methodology was “oh yeah and we didn’t bother normalizing it”

                              1. 3

                                Hi Hillel! I’m glad you found this, and thank you for taking the time to respond.

                                I’m not sure you necessarily need to mount a defense, either. I didn’t consciously intend to set your talk up as the antagonist in my post, but I realize this is sort of what I did. The attitude I’m trying to refute (that empirical science is the only source of objective knowledge about software) is somewhat more extreme than the position you advocate. And the attitude you object to (that software “can’t be studied” empirically, and nothing can be learned this way) is certainly more extreme than the position I hoped to express. I think in the grand scheme of things we largely share the same values, and our difference of opinion is rather esoteric and mostly superficial. That doesn’t mean it’s not interesting to debate, though.

                                Re: Omitted variable bias

                                You seemed to suggest that research could account for omitted variable bias by “cross-correlating” studies

                                • across different companies
                                • within one same company before and after adopting/disadopting the practice
                                • across different teams within the same company.

                                I submit to you this is not the case. Continuing with the CD example, suppose CD doesn’t improve outcomes but the “trendiness” that leads to it does. It is completely plausible for

                                • trendy companies to be more likely to adopt CD than non-trendy companies
                                • trendy teams within a company to be more likely to adopt CD than non-trendy teams
                                • a company that is becoming more trendy is more likely to adopt CD and be trendier before the adoption than after adoption
                                • a company that is becoming less trendy is more likely to disadopt CD and be trendier before the disadoption than after

                                If these hold, then all of the studies in the “cross-correlation” you describe will still misattribute an effect to CD.

                                You can’t escape omitted variable bias just by collecting more data from more types of studies. In order to legitimately address it, you need to do one of:

                                • Find some sort of data that captures “trendiness” and include it as a statistical control.
                                • Find an instrumental variable
                                • Find data on teams within a company that were randomly assigned to CD (so that trendiness no longer correlates with the decision to adopt).

                                If you don’t address a plausible omitted variable bias in one of these ways, then basically you have no guarantee that the effect (or lack of effect) you measured was actually the effect of the practice and not the effect of whatever social conditions or ideology led to the adoption of your practice (or something else that those social conditions caused). This is a huge threat to validity, especially to “code mining” studies whose only dataset is a git log and therefore have no possible hope of capturing or controlling the social or human drivers behind the practice. To be totally honest, I assign basically zero credibility to the empirical argument of any “code mining” study for this reason.

                                Re: The analogy to medicine

                                As @notriddle seemed to be hinting at, professions comprehensively guided by science are the exception, not the rule. Science-based lawyering seems… unlikely. Science-based education is not widely practiced, and is controversial in any case. Medicine seems to be the major exception. It’s worth exploring the analogy/disanalogy between software and medicine in greater detail. Is software somehow inherently more difficult to study than medicine?

                                Maybe not. You brought up two good points about avenues of software research.

                                Companies do internal investigations on this- Microsoft and IBM especially has a lot of great work in this style.

                                and

                                In some cases, companies are willing to instrument themselves- see TSP or the NASA data sets.

                                I think analysis of this form is miles more persuasive than computer lab studies or code mining. If a company randomly selects certain teams to adopt a certain practice and certain teams not to, this solves the realism problem because they are, in fact, real software teams. And it solves the omitted variable bias problem because the practice was guaranteed to have been adopted randomly. I think much of the reason medicine has been able to incorporate empirical studies so successfully is because hospitals are so heavily “instrumented” (as you put it) and willing to conduct “clinical trials” where the treatment is randomly assigned. I’m quite willing to admit that we could learn a lot from empirical research if software shops were willing to instrument themselves as heavily as hospitals, and begin randomly designating teams to adopt practices they want to study. I think it’s quite reasonable to advocate for a movement in that direction.

                                But whether or not we should advocate for more better data/more research is orthogonal to the main concern of my post: in the meantime, while we are clamoring for better data, how ought we evaluate software practices? Do we surrender to nihilism because the data doesn’t (yet) paint a complete picture? Do we make wild extrapolations from the faint picture the data does paint? Or should we explore and improve the body of “philosophical” ideas about programming, developed by programmers through storytelling and reflection on experience?

                                It is very important to do that last thing. I wrote my post because, for a time, my own preoccupation with the idea that only scientific inquiry had an admissible claim to objective truth prevented me from enjoying and taking e.g. “A Philosophy of Software Design” seriously (because it was not empirical), and realizing what a mistake this was was somewhat of a personal revelation.

                                Re: Epistemology

                                Science is about admitting that we are going to make mistakes, that we’re going to naturally believe things that aren’t true, no matter how mentally rigorous we try to be. That’s what makes it so important and so valuable. It gives us a way to say “well you believe X and I believe not X, so which is it?”

                                Science won’t rescue you from the fact that you’re going to believe things that aren’t true, no matter how mentally rigorous you try to be. Science is part of the attempt to be mentally rigorous. If you aren’t mentally rigorous and you do science, your statistical model will probably be wrong, and omitted variable bias will lead you to conclude something that isn’t true.

                                Science, to me, is merely a toolbox for generating persuasive empirical arguments based on data. It can help settle the debate between “X” and “not X” if there are persuasive scientific arguments to be found for X, and there are not persuasive scientific arguments to be found for “not X” – but just as frequently, there turn out to be persuasive scientific arguments for both “X” and “not X” that cannot be resolved empirically must be resolved theoretically/philosophically. (Or – as I think describes the state of software research so far – there turn out to be persuasive scientific arguments for neither “X” nor “not X”, and again, the difference must be resolved theoretically/philosophically).

                                [Being empirical]… means reading dozens of papers that are all maddeningly contradictory. It means going through papers agonizingly carefully because the entire thing might be invalidated by an offhand remark.[1] It means reading paper’s references, and the references’ references, and trawling for followup papers, and reading the followup paper’s other references.

                                That’s a way more nuanced claim than just “code review works!” And it’s way, way more nuanced than about 99% of the Code Review takes I see online that don’t talk about the evidence. Empiricism means being more diligent and putting in more work to understand, not less.

                                I value this sort of disciplined thinking – but I think it’s a mistake to brand this as “science” or “being empirical”. After all, historians and philosophers also agonize through papers, crawling the reference tree, and develop highly nuanced, qualified claims. There’s nothing unique to science about this.

                                I think we should call for something broader than merely disciplined empirical thinking. We want disciplined empirical and philosophical/anecdotal thinking.

                                My ideal is that software developers accept or reject ideas based on the strength or weakness of the argument behind them, rather than whims, popularity of the idea, or the perceived authority or “charisma” of their advocates. For empirical arguments, this means doing what you described – reading a bunch of studies, paying attention to the methodology and the data description, following the reference trail when warranted. For philosophical/anecdotal arguments, this means doing what I described – mentally searching for inconsistencies, evaluating the argument against your own experiences and other evidence you are aware of.

                                Occasionally, this means the strength of a scientific argument must be weighed against a philosophical/anecdotal argument. The essence of my thesis is that, sometimes, a thoughtful, well-explained story by a practitioner can be a stronger argument than an empirical study (or more than one) with limited data and generality. “X worked for us at Dropbox and here is my analysis of why” can be more persuasive to a practitioner than “X didn’t appear to work for undergrad projects at 12 institutions, and there is not a correlation between X and good outcome Y in a sampling of Github Repos”.

                                1. 2

                                  Hi, thanks for responding! I think we’re mostly on the same page, too, and have the same values. We’re mostly debating the degrees and methods of here. I also agree that the issues you raise make things much more difficult. My stance is just that while they do make things more difficult, they don’t make it impossible, nor do they make it not worth doing.

                                  Ultimately, while scientific research is really important, it’s only one means of getting knowledge about something. I personally believe it’s an incredibly strong form- if philosophy makes one objective claim and science makes another, then we should be inclined to look for flaws in the philosophy before looking for flaws in the science. But more than anything else, I want defence in depth. I want people to learn the science, and the history, and the philosophy, and the anthropology, and the economics, and the sociology, and the ethics. It seems to me that most engineers either ignore them all, or care about only one or two of these.

                                  (Anthro/econ/soc are also sciences, but I’m leaving them separate because they usually make different claims and use different ((scientific!)) than what we think of as “scientific research” on software.)

                                  One thing neither of us have brought up, that is also important here: we should know the failure modes of all our knowledge. The failure modes of science are really well known: we covered them in the article and our two responses. If we want to more heavily lean on history/philosophy/anthropology, we need to know the problems with using those, too. And I honestly don’t know them as well as I do the problems with scientific knowledge, which is one reason I don’t push it as hard- I can’t tell as easily when I should be suspicious.

                                2. 3

                                  What a fantastic response.

                                  When doctors get involved in fields such as medical education or quality improvement and patient safety, they often have a similar reaction to Richard’s. The problem is in thinking that the only valid way to understand a complex system is to study each of its parts in isolation, and if you can’t isolate them, then should just give up.

                                  As Hillel illustrated nicely here, you can in fact draw valid conclusions from studying “complex systems in the wild”. While this is a “messier” problem, it is much more interesting. It requires a lot of creativity but also more rigor in justifying and selecting the methodology, conducting the study, and interpreting the results. It is very easy to do a subpar study in those fields, which confounds the perception about the fields being “unscientific”.

                                  A paper titled Research in the Hard Sciences, and in Very Hard “Softer” Domains by Phillips, D. C. discusses this issue. Unfortunately, it’s behind a paywall.

                                  1. 3

                                    Can we study teachers? Can we study doctors and nurses?

                                    The answer to that question might be “no”.

                                    When you’re replying to an article that’s titled “The False Promise of Science”, with a bunch of arguments against empirical software engineering that seem applicable to other fields as well, and your whole argument is basically an analogy, you should probably consider the possibility that Science is Just Wrong and we should all go back to praying to the sun.

                                    The education field is at least as fad- and ideology-driven as software, and the medical field has cultural problems and studies that don’t reproduce. Many of the arguments given in this essay are clearly applicable to education and medicine (though not all of them obviously are, I can easily come up with new arguments for both fields). The fundamental problem with applying science to any field of endeavor is that it’s anti-situational at the core. The whole point of The Scientific Method is to average over all but a few variables, but people operating in the real world aren’t working with averages, they’re working with specifics.

                                    The argument that software isn’t special cuts both ways, after all.


                                    I’m not sure if I actually believe that, though.

                                    The annoying part about this is that, as reasonably compelling as it’s possible to make the “science sucks” argument sound, it’s not very conducive to software engineering, where the whole point of the practice is to write generalized algorithms that deal with many slight variants of the same problem, so that humans don’t have to be involved in every little decision. Full-blown primativism, where you reject Scalable Solutions(R) entirely, has well-established downsides like heightened individual risk; one of the defining characteristics of modernism is risk diffusion, after all.

                                    Adopting hard-and-fast rules is just a trade-off. You make the common case simpler, and you lose out in the special cases. This is true both within the software itself (it’s way easier to write elegant code if you don’t have weird edge cases) and with the practice. The alternative, where you allow for exceptions to the rules, is decried as bad for different reasons.

                                    1. 6

                                      That is absolutely a valid counterargument! In response, I’d like to point out that we have learned a lot about those fields! Just a few examples:

                                      I’m don’t know very much about classroom teaching or nursing, so I can’t deep-dive into that research as easily as I can software… but there are many widespread and important studies in both fields that give us actionable results. If we can do that with nursing, why not software?

                                      1. 1

                                        To be honest, I think you’re overselling what empirical science tells us in some of these domains, too. Take the flipped classroom one, since it’s an example I’ve seen discussed elsewhere. The state of the literature summarized in that post is closer to: there is some evidence that this might be promising, but confidence is not that high, particularly in how broadly this can be interpreted. Taking that post on its own terms (I have not read the studies it cites independently), it suggests not much more than that overall reported studies are mainly either positive or inconclusive. But it doesn’t say anything about these studies’ generalizability (e.g. whether outcomes are mediated by subject matter, socioeconomic status, country, type of institution, etc.), suggests they’re smallish in number, suggests they’ve not had many replication attempts, and pretty much outright says that many studies are poorly designed and not well controlled. It also mentions that the proxies for “learning” used in the studies are mostly very short-term proxies chosen for convenience, like changes in immediate test scores, rather than the actual goal of longer-term mastery of material.

                                        Of course that’s all understandable. Gold-standard studies like those done in medicine, with (in the ideal case) some mix of preregistration, randomized controlled trials, carefully designed placebos, and longitudinal follow-up across multi-demographic, carefully characterized populations, etc., are logistically massive undertakings, and expensive, so basically not done outside of medicine.

                                        Seems like a pretty thin rod on which to hang strong claims about how we ought to reform education, though. As one input to qualitative decision-making, sure, but one input given only its proper weight, in my opinion significantly less than we’d weight the much better empirical data in medicine.

                                    2. 2

                                      Dammit, man. That was a great response. I don’t think I’ll ever comment anything anywhere just so my comment won’t be compared to this.

                                      1. 1

                                        My favorite software book is Data and Reality, which is a philosophical investigation into the nature of information representation.

                                        A beautiful book, one of my favorites as well.

                                        rest of post….

                                        While I thought the article articulated something important which I agree with, its conclusion felt a bit lazy and too optimistic for my taste – I’m more persuaded by the POV you’ve articulated above.

                                        While we’re making analogies, “writing software is like writing prose” seems like a decent one to explore, despite some obvious differences. Specifically relevant is the wide variety of different and successful processes you’ll find among professional writers.

                                        And I think this explains why you might be completely right that something like TDD is valuable for you, even though empirical studies don’t back up that claim in general. And I don’t mean that in a soggy “everyone has their own method and they’re all equally valid” way. I mean that all of your knowledge, the way think about programming, your tastes, your knowledge of how to practice TDD in particular, and on and on, are all inputs into the value TDD provides you.

                                        Which is to say: I find it far more likely that TDD (or similar practices with many knowledgeable, experienced supporters) have highly context sensitive empirical value than none at all. I don’t foresee them being one day unmasked by science as the sacred cows of religious zealots (though they may be that in some specific cases too).

                                        For something like TDD, the “treatment” group would really need to be something like “people who have all been taught how to do it by the same expert over a long enough time frame and whose knowledge that expert has verified and signed off on.”

                                        I’m not shilling for TDD, btw – just using it as a convenient example.

                                        The broader point is that effects can be real but extremely hard to show experimentally.

                                        1. 1

                                          “We’re not basing our worldview off a single study. We’re doing many of them, in many different contexts, to get different facets of what the answer might actually be.”

                                          That’s exactly what I do for the sub-fields I study. Especially formal proof which I don’t understand at all. Just constantly looking at what specialists did… system type/size, properties, level of automation, labor required… tells me a lot about what’s achievable and allows mix n’ matching ideas for new, high-level designs. That’s without even needing to build anything which takes a lot longer. That specialists find the resulting ideas worthwhile proves the surveys and integration strategy work.

                                          So, I strongly encourage people to do a variety of focused studies followed by integrated studies on them. They’ll learn plenty. We’ll also have more interesting submissions on Lobsters. :)

                                          “When someone says “you are unprofessional if you don’t use TDD” or “Dynamic types are immoral”, I know, with scientific certainty, that they don’t actually know. “

                                          I didn’t think about that angle. Actually, you got me thinking maybe we can all start telling that to new programmers. They get warned the field is full of hype, trends, etc that usually don’t pan out over time. We tell them there’s little data to back most practices. Then, experienced people cutting them down or getting them onto new trend might have less effect. Esp on their self-confidence. Just thinking aloud here rather than committed to idea.

                                          “Science is about admitting that we are going to make mistakes”

                                          I used to believe science was about finding the truth. Now I’d go further than you. Science assumes we’re wrong by default, will screw up constantly, and are too biased or dishonest to review the work alone. The scientific method basically filters bad ideas to let us arrive a beliefs that are justifiable and still might be wrong. Failure is both normal and necessary if that’s the setup.

                                          The cognitive dissonance make it really hard like you said. I find it a bit easier to do development and review separately. One can be in go mode iterating stuff. At another time, in skeptical mode critiquing the stuff. The go mode also gives a mental break and/or refreshes the mind, too.

                                          1. 1

                                            You’d be surprised. “Two comparisons of programming languages”, in “making software”, does this with nine teams (but only for one day).

                                            My reading (which is congruent with my experiences) indicates a newly-put-together team takes 3-6 months before productivity stabilizes. Some schools of management view this as ‘stability=groupthink, shuffle the teams every 6 months’ and some view it as ‘stability=predictability, keep them together’. However, IMO this indicates to me that you might not be able to infer much from one day of data.

                                            1. 2

                                              To clarify, that specific study was about nine existing software teams- they came to the project as a team already. It’s a very narrow study and definitely has limits, but it shows that researchers can do studies on teams of professionals.

                                            2. 1

                                              People bring up how expensive it is. And it is expensive, just as it’s expensive to study people in general. But here’s the thing. We are one of the richest industries in the history of the world. Apple’s revenue last year was a quarter trillion dollars. That’s not something we should leave to folklore and feelings. We’re worth studying.

                                              I don’t think I understand what you’re saying. Software is expensive, and for some companies, very profitable. But would it really be more profitable if it were better studied? And what exactly does that have to do with the kinds of things that the software engineering field likes to study, such as defect rates and feature velocities? I think that in many cases, even relatively uncontroversial practices like code review are just not implemented because the people making business decisions don’t think the prospective benefit is worth the prospective cost. For many products or services, code quality (however operationalized) makes a poor experimental proxy for profitability.

                                              Inasmuch as software development is a form of industrial production, there’s a huge body of “scientific management” literature that could potentially apply, from Frederick Taylor on forward. And I would argue it generally is being applied too: just in service of profit. Not for some abstract idea of “quality”, let alone the questionable ideal of pure disinterested scientific knowledge.

                                              1. 1

                                                Mistakes are becoming increasingly costly (e.g., commercial jets falling from the sky) so understanding the process of software-making with the goal of reducing defects could save a lot of money. If software is going to “eat the world”, then the software industry needs to grow up and become more self-aware.

                                                1. 1

                                                  Aviation equipment and medical devices are already highly regulated, with quality control processes in place that produce defect rates orders of magnitude less than your average desktop or business software. We already know some things about how to make high-assurance systems. I think the real question is how much of that reasonably applies to the kind of software that’s actually eating the world now: near-disposable IoT devices and gimmicky ad-supported mobile apps, for example.

                                            1. 2

                                              Please educate a Chrome hater: does it not allow installing extensions from source by hand at all, or the procedure is just too complex for a casual user?

                                              1. 4

                                                That doesn’t matter. Most people don’t, and shouldn’t, instead add-ons from outside the chrome store. Too much chrome-targeted malware floating around.

                                                Also, if Google starts blocking ad blockers from the store, then they are probably going to go ahead and break them at the API level, too. I was willing to give them the benefit of the doubt with the declarative network API, but actually refusing to provide ublock origin is too much.

                                                1. 2

                                                  By this analogy most people shouldn’t install Windows software from outside the Windows Store because there’s too much malware for Windows floating around, which would imply they shouldn’t install Firefox, GIMP, or OpenVPN.

                                                  1. 3

                                                    Yes, Windows does have a serious malware problem. I’m glad that you’re willing to engage in a conversation on how we can arrange for simple rules that everyone can follow to ensure that any software they install has been vetted, since not everyone can read code, and even people who can don’t necessarily have the time to read all of it. A nice ideal, simple solution would be “never install software from outside the Windows Store.” Too bad too much good software predates the Windows Store for that to be practical, and Microsoft’s rules are way too strict.

                                                    1. 3

                                                      In the real world there are also many people taking advantage of people’s inexperience and inattention. Does it mean we should limit people’s ability to make deals and sign contracts? The solution is to educate people about the ways malicious people can screw them up and punish malicious people. And improve the underlying software of course.

                                                      Why in the software industry we should create nanny states that tell people what they can and cannot do? App stores operators can’t and don’t read all the source code either. They are not even willing (or unable) to actually have a human look at all submissions, if they resort to automated reviews and replies. App store operators can themselves do malicious things like inserting adware and spyware, and if people don’t read their user agreements carefully, they can legally get away with it (or if the platform is locked down and users have no alternative, they may have to put up with it even after reading the agreement).

                                                      1. 3

                                                        Does it mean we should limit people’s ability to make deals and sign contracts?

                                                        There are limits on people’s ability to make contacts. Most nations don’t allow you to sign away your right to vote, for example. And that’s a good thing, because such contracts are invariably coerced.

                                                        But that really isn’t relevant here, since I don’t actually support making side-loading illegal, just discouraging it.

                                                        The solution is to educate people about the ways malicious people can screw them up and punish malicious people.

                                                        Too bad it’s so hard to trace people’s identities on the internet, so that they can be effectively punished. That’s why Google collects like $2 for a Play Store licence; most people can pay it, very few can pay for it millions of times if they keep getting kicked off.

                                                        As for education… What do you even teach? Even professionals fall for phishing attacks.

                                                        And improve the underlying software of course.

                                                        Let’s do that, too.

                                                        Why in the software industry we should create nanny states that tell people what they can and cannot do?

                                                        Harm reduction. I’m guessing that’s a rhetorical question, since the answer is so obvious, and is identical for all “nanny state” initiatives.

                                                        App stores operators can’t and don’t read all the source code either. They are not even willing (or unable) to actually have a human look at all submissions, if they resort to automated reviews and replies.

                                                        Still better than the wild web.

                                                        App store operators can themselves do malicious things like inserting adware and spyware, and if people don’t read their user agreements carefully, they can legally get away with it

                                                        That’s been possible the whole time, and it’s been done since at least Windows XP and the so-called Genuine Windows Install gadget that they shipped through Windows Update. If your OS vendor is malicious, you’re screwed either way.

                                                2. 3

                                                  It’s actually pretty straightforward.You just have to enable a developer mode and select a directory. These days, you can’t expect (capable) users to do so, though. Times have changed. I’m speaking from my own perspective; installing and in particular, updating, extensions manually isn’t something I’d be happy to do, nor is it something I’d recommend to most people.

                                                  I’ve been using Chrome for a few years now. Prior to that I’d used Firefox and it was a rather painful experience because of the crashes and incompatible changes. And even before that, I used Opera 12.x. I gave up after a year or so without (Linux) updates and never came back. I thought they would just replace the rendering core and keep the UI, but they completely rebuilt the browser in a way that made me think I could just use Chrome.

                                                  Why do I use Chrome over Chromium? Well, that’s a good question. I switched from Chromium to Chrome after some crazy Debian developer decided to compile it with extensions disabled and I found myself in a position where I needed to jump into a new working day and had to fix my browser ASAP.

                                                  I used to believe in the power of free market, and I still do. The problem with free market is it only works given a reasonably large competition. The situation we witness regarding the web browsers today is anything but that. The technology is so crazy complex it’s not a viable option to create a competing browser from scratch, and that’s just what Google wants. Modern web is a technological tragedy and I wouldn’t be surprised if Google intentionally supported its adoption and the trend to make it even more powerful (and complicated) in general just so they can leverage it and use it to their advantage in the end.

                                                  I probably will consider giving Firefox (or Firefox-based browser, such as Palemoon) an another chance, and will probably setup Syncthing so that I can cancel my Google Play Music subscription (unlike Spotify, it allows to upload music and share it between devices; at least that’s how it was the last time I checked). The thing is, I’m becoming more and more sceptical this is good for anything. Majority of users doesn’t care. It’s really sad I have to say this, but regulatory bodies (for example in the EU) might be the best cards we hold in our hand. As a devoted libertarian, this troubles me deeply.

                                                  What alternatives do you (people) see? Trying to convince as many people as possible that Google is evil and they are morally obligated to switch to anything else? When Mozilla does something stupid, do that again? Well, maybe. I still want to believe this is the way to go, and it’s what I’ve been trying to do in the past, but there’s only so much discomfort one can handle. Majority of people doesn’t care, doesn’t want to care and will not care. That’s what I value people like RMS for, as much as I don’t like him on a personal level: standing behind his opinions is more valuable for him than a comfort he could enjoy by simply following the crowd.

                                                  1. 1

                                                    I’m not 100% sure but I think they removed being able to install extensions manually to prevent malware. I think you can still do it on a domain with GPOs.

                                                    I think it’s a good thing. A lot of apps try to install extensions without your consent. Not sure what Google could do to allow non-entreprise users to install extensions manually while preventing malware to do so.

                                                  1. 4

                                                    In a potentially far-reaching move […]

                                                    There’s nothing “potentially” about it. I don’t know why it would be a good thing for the government to 1) decide what constitutes software accessibility, and 2) force people by threat of legal action to change the way their websites work when there’s no negative externality to not being able to use a website

                                                    1. 12

                                                      How is this any different than the government mandating certain architectural/design rules for commercial buildings and public spaces (ADA)?

                                                      1. 3

                                                        That’s the whole point if I understand the case correctly (and I may not! I’m not a lawyer!).

                                                        See here:

                                                        First, the Ninth Circuit reaffirmed its position that, to be covered by the ADA, a website or mobile app must have a nexus to a physical place of public accommodation. The court stated that this nexus was “critical” to its analysis in the Domino’s case where the “alleged inaccessibility of Domino’s website and app impedes access to the goods and services of its physical pizza franchises – which are places of public accommodation.” The Ninth Circuit said in a footnote that it was not deciding whether “the ADA covers the websites or apps of a physical place of public accommodation where the inaccessibility does not impede access to the goods and services of a physical location.”

                                                        Like, that’s the key thing–the website augments a physical location.

                                                      2. 9

                                                        there’s no negative externality to not being able to use a website

                                                        If an insufficient number of websites include accurate accessibility metadata, browser developers won’t write code to consume it. If nobody uses it, then the effort will never get off the ground, and accessibility tool (and, almost identically, search engines) rely on heuristics instead. The benefit is reaped by the web authors, who don’t have to write the metadata, but is born by browser and search engine developers, who are not direct parties. Thus, it’s an externality.

                                                        1. 2

                                                          Allow me to clarify, I’m looking at this in the frame of actions that are legally recognized as externalities; I think the example you point out is a cultural/social consequence of adhering to accessibility standards. Legally, negative externalities are generally effects that directly cause harm to a party not involved (I’m sure there are exceptions, but we’re talking about the rule here). If a website doesn’t work, that is neither endangering the person unable to access the website, nor is directly inflicting harm to them. That’s why I’m saying the government shouldn’t really be involved in something like this.

                                                          1. 2

                                                            In the case of Dominos there was an online-only promotion that was inaccessible, so they literally lost money if they had to phone in an order.

                                                            1. 2

                                                              I think calling it losing money is overzealous. It would be losing money if they had no choice but to order from Dominos. They weren’t forced to order from Dominos. If there’s a coupon for groceries that gets mailed out by a grocery store, we don’t legally pursue the grocery store for being exclusionary if someone that doesn’t have a mailbox didn’t get the coupon (e.g. homeless folks). They didn’t lose money, they just didn’t save some money; those aren’t the same thing

                                                              1. 1

                                                                Well of course folks who aren’t customers aren’t going to care/lose money, but there was literally an unfair financial advantage in favor of those who could use the website to order pizza vs those who could not use the website to order pizza. Folks who had to use a phone to order pizza literally paid more than those who could use the website. They lost money.

                                                        2. 7

                                                          In addition to what other posters have raised - IMO ‘no negative externality’ is a defensible claim, but far from a sure one.

                                                          The negative externality of impeded access is paid by the carers (usually family), who end up spending their time managing the affairs of someone who would otherwise be able to do so themselves.

                                                          1. 5

                                                            I think you’ve misunderstood the ADA. The whole point of creating a private right of action is that the government does not set specific standards. Instead those affected by inaccessible accommodations sue, and a court decides if the place is in fact inaccessible. Accessibility is the standard. Places of business are free to meet that standard in any way that actually meets it.

                                                            1. 2

                                                              You’re right, I did misunderstand it.

                                                            2. 6

                                                              I’ve also tried to make this point but nobody wants to hear it… I don’t think we want the government to get involved in UI/UX design.

                                                            1. 31

                                                              I’d like to comment on another meta-point in the article.

                                                              The trouble with this type of platform restriction is that the opinions do not go away. Those who are removed from social media platforms often feel ostracized, angry and perhaps even vindicated in their persecution. They take to other platforms like Gab and Voat, where other like minded people validate those opinions. They leave larger Internet communities with a variety of voices that could potentially steer their own opinions in a more moderate direction.

                                                              This was a perfectly reasonable and effective position on content moderation until recently. But what we’ve learned about internet communities in, say, the past decade, is that sunlight is not always the best disinfectant. Trolls and Nazis and etc. will reliably ruin platforms if left unchecked, and even swing moderates into their camp; the idea that they can be made more civil by exposure to cultural norms is simply not borne out by the evidence. Consequently this sort of free speech idealism is naïve to the point of being unethical. Free speech isn’t an unimpeachable virtue, or some end to work towards. It’s a means, a tool, that we’re obliged to wield to just ends.

                                                              Furthermore, getting the opinions to go away isn’t really the goal. Laws don’t make crime disappear, but we still have them, because they tend to have positive outcomes on their societies. Similarly, deplatforming doesn’t make bad ideas disappear, but it does reduce their availability and accessibility. Deplatforming works, let’s keep doing it.

                                                              1. 11

                                                                Does Deplatforming work and what do you mean by work? Brendan O’Neill has some very good points about how things we currently consider ‘progressive’ have been deplatformed in previous centuries. https://www.youtube.com/watch?v=BtWrljX9HRA

                                                                Furthermore, I’d suggest reading The Coddling of the American Mind, which talks a lot about the current call-out culture in academia, that leads to harming the relationship between students and professors; preventing people from being able to discuss difficult topics and ideas without fear of retribution or being called Nazis or White Suprematists.

                                                                Trolls and Nazis and etc. will reliably ruin platforms if left unchecked, and even swing moderates into their camp

                                                                One thing I didn’t really cover is the issue with anonymity. That is another problem space (and I’m working on a full post on it). Anonymous networks are really … interesting … as far as content (4chan, 8ch and other chans .. Reddit/Voat/HackerNews, ActivityPub/Fediverse stuff). People act very different anonymously, which is one reason Facebook and Google+ pushed so much for only having real names/people, and why Reddit/Twitter require so much moderation to make them more (advertiser) “friendly” platforms. There are a lot of complexities there to unpack.

                                                                1. 7

                                                                  If you get to link to YouTube and pop politics books, then I get to link to https://slatestarcodex.com/2017/05/01/neutral-vs-conservative-the-eternal-struggle/ and https://slatestarcodex.com/2015/08/15/my-id-on-defensiveness/ which makes a pretty reasonable argument that there is no way Voat could have possibly gone right.

                                                                  Or, to summarize it another way, the same way the distinction between consumer tech and enterprise tech doesn’t exist, the distinction between “separate online communities” doesn’t exist either. Stuff that happens on one will have an effect on the other, inevitably. The discourse on Twitter (including the effects of their algorithms) leaks onto Lobsters and back onto Twitter again; you can have some control over your little corner, but you aren’t actually separate.

                                                                  1. 16

                                                                    Does Deplatforming work and what do you mean by work?

                                                                    By “work” I primarily mean that fewer people get exposed to hate speech at a macro scale, especially inadvertently. But also that fewer people get recruited into hate groups, especially for the lulz. And also that hate speech propagandists, robbed of some of the dopamine from engagement on larger platforms, are discouraged from continuing. And yes, all evidence suggests that deplatforming works by these metrics.

                                                                    People act very different anonymously,

                                                                    Again, this was a truism like 10 years ago, but we’ve since learned that, anonymous or not, the internet tends to create echo-bubble environments that bring out the most extreme and frequently negative properties of the human condition. There’s an abundance of grotesque, racist, whatever nonsense written by people on Facebook next to their real names. There aren’t any consequences for it, really, so why not?

                                                                    1. 7

                                                                      By “work” I primarily mean that fewer people get exposed to hate speech at a macro scale, especially inadvertently

                                                                      I think this gets into dangerous territory. We should be exposed to things we don’t like or agree with. Having friends of different political backgrounds and ideological persuasions, and honestly talking about tough issues, is how we grow and change over time. I’m not for bullying, but I’m also not for safetism. It’s a hard line to cute and much harder on-line than in real life. Like the Brendan O’Neill debate I posted, there was a time when people who thought homosexually wasn’t wrong or that we didn’t need god or that the Bible should be translated into languages that could be read by everyone, were de-platformered, marginalized and told their ideas were greatly offensive. To say which ideas are good or bad for society change greatly over time. I know my views on what is just and unjust have changed significantly from my 20s to my 30s.

                                                                      Yes there are trolls who just shit post. But there are also a lot of true believers, who went cut from a platform they feel they’re making reasonable comments on, will go further into their cause and more radical. We saw that when Anita Sarkeesian deleted all the YouTube comments on her videos and locked them. Yes there were typical garbage YouTube comments, but there were also a lot of reasonable arguments. You delete all of those, and people tend to go harder in and be less reasonable. De-platforming lets people grab onto the same victimhood culture as those who de-platform; the “my views are being oppressed” rubbish instead of “let’s talk about things and maybe agree to disagree.”

                                                                      I think I understand where your coming from though. I think these topics are pretty complex though, and they can get into some really gritty details, for example the recent Stack Exchange / pronoun / code of conduct fiasco. Those are the type of debates that quickly get muted everywhere because we’re simply to afraid to have them. They then show up as much more polarized and much more extreme hard left/right lines when they appear on Reddit/Gab/Voat/etc.

                                                                      1. 16

                                                                        We should be exposed to things we don’t like or agree with.

                                                                        Sometimes yes, sometimes no.

                                                                        It’s fine to say that Chicago-school economists should be exposed to Austrian economic theory. Or that Baptists should be exposed to Lutheran theology. That Ford owners should be exposed to GM fans. That NIMBYs should be exposed to YIMBYs.

                                                                        It’s not fine to say that a rape survivor should be exposed to the gloating of their assailant after being found not guilty on a technicality. Or that a black school child should be exposed to a Klu Klux Klan rally on their walk home from school. These things are certainly and technically “different ideological persuasions” but no good is advanced by enduring them.

                                                                        So there’s definitely a line where the ideal of free speech, or the marketplace of ideas, or whatever, is insufficient to justify the outcome. We’re just debating where that line is.

                                                                        It used to be that we could talk about white supremacy or Nazis or whatever pretty freely, because nobody (or very very few people) were actually threatened by those things. But the context has changed, white supremacists are marching in our streets with literal torches, and lots of people have very good reason to be afraid of what might come next. The line of what’s acceptable to deal with, in this particular space, has moved. So, no, at a societal level, we shouldn’t be forced to confront this particular “thing we don’t like or agree with” in deference to an abstract ideal. We are justified in stomping it out, like an immune system response, with tools like deplatforming, and whatever others are effective.

                                                                        1. 16

                                                                          we’re simply to afraid to have [debates]

                                                                          This is not a fact. This is a right-wing trope that’s not based on reality all that much.

                                                                          No one is “afraid of debate”. Actually people are just tired of having to prove that they deserve to exist, to be themselves, to love who they love, and so on. These things should not be up for debate.

                                                                          De-platforming lets people grab onto the same victimhood culture as those who de-platform

                                                                          They grab onto that either way.


                                                                          Highly recommended listening:

                                                                          1. 15

                                                                            We should be exposed to things we don’t like or agree with. Having friends of different political backgrounds and ideological persuasions, and honestly talking about tough issues, is how we grow and change over time.

                                                                            Fascists don’t argue in good faith. You aren’t going to change minds in a positive direction by platforming them. What you will do is tacitly promote the idea that genocide is a valid topic of disagreement, and help them recruit.

                                                                            Deplatforming them works.

                                                                            1. 4

                                                                              Milo wasn’t wrecked by deplatforming. Milo was wrecked by defending pedophilia and directly working with neo-Nazis, which is what made his right-wing supporters turn on him. The “Deplatforming stopped Milo” narrative only appeared like a year later.

                                                                              1. 3

                                                                                The “Deplatforming stopped Milo” narrative only appeared like a year later.

                                                                                At the exact point that Milo said that he no longer had an audience enough to sustain him, and had to work on other projects for money.

                                                                                You’re saying that he was “wrecked by defending pedophilia and directly working with neo-Nazis, which is what made his right-wing supporters turn on him”, which is in and of itself, a form of deplatforming. Whether or not he did it himself is irrelevant to the fact of it being deplatforming or not. It’s like saying “he didn’t drive a vehicle, he drove a truck”.

                                                                                1. 1

                                                                                  I believe that those incidents are what motivated his deplatforming, and the decline in audience he suffered was multiplied by his loss of access to a large platform.

                                                                                2. 1

                                                                                  Fascists don’t argue in good faith.

                                                                                  And everyone is a fascist who doesn’t agree to your agenda. You can be “deplatformed” from the largest mastodon instance if you have the “wrong opinion” on funding domestic terrorist organizations (the antifa), and voice it.

                                                                                  1. 1

                                                                                    You can be “deplatformed” from the largest mastodon instance if you have the “wrong opinion” on funding domestic terrorist organizations (the antifa), and voice it.

                                                                                    If I’m reading between the lines correctly, here, and the implication is that you think a group literally called Anti-Fascists are terrorists, then I don’t think you really get to call foul when people judge that to be roughly aligned with fascism, eh?

                                                                                    1. 0

                                                                                      there was a poll:

                                                                                      • you support the antifa (that is a terrorist organization in the USA!) with money
                                                                                      • you are a fascist

                                                                                      I think the antifa an their supporters are the fascists of these days. The binary rhetoric, the violent opression of different opinions, etc. are just as bad as what they claim to be against.

                                                                                      Regarding de-platforming: I was born in a communist dictatorship. Lots of voices and opinions were “deplatformed”, in the name of the greater good, “antifascism”. For example punk music, and punks, who are now thought to be a left wing/left leaning genre, were just as much enemies of the “left wing” state… I believe discourse is necessary and nobody should be de-platformed, as long as their actions are legal, and when they are illegal, they should be regardless of political stance.

                                                                                      1. 1

                                                                                        I think the antifa an their supporters are the fascists of these days. The binary rhetoric, the violent opression of different opinions, etc. are just as bad as what they claim to be against.

                                                                                        Well, that’s ludicrous.

                                                                                        1. 1

                                                                                          That’s also an opinion, and I’m glad to hear that. Now I won§’t go to de-platform you for disagreeing with me. It should be this simple. Unfortunately it is not.

                                                                                    2. 0

                                                                                      Case in point.

                                                                                3. 1

                                                                                  By “work” I primarily mean that fewer people get exposed to hate speech at a macro scale, especially inadvertently.

                                                                                  Personally I’m totally uncertain on this topic, but seeing that banning people from reddit has made them relocate to voat, banning threads on 4chan has made them relocate on infinitychan. There they gather, organise, produce more propaganda and create more stories. Would they have done so on the previous platform? probably. What I don’t know is if it would be better or worse. What I find even more perplexing is that if one, “edgy”, community gets band on one site, it gives a push to all of them. Ban racists on facebook, and reddit will use it to push their narrative.

                                                                                  I really don’t see a solution, but what’s wrong it to claim that deplatforming is a step forward. That’s like saying that just throwing your rubbish out of the window is fine, instead of putting it in the recycling bin.

                                                                                  1. 5

                                                                                    Banning people from reddit has made them relocate to voat, banning threads on 4chan has made them relocate on infinitychan

                                                                                    I don’t care about the true believers. Let them fester in their holes. I care about the thousands or millions of passersby, regular visitors to popular sites like Reddit or (less so) 4chan, who get exposure to these hate cultures when comments by the trolls are co-mingled with rational people in unrelated articles, or when racist memes are mixed in with cat videos on /r/all. Reducing that exposure is a huge net win and worth doing.

                                                                                    1. 2

                                                                                      But that’s my point. You just need a few “true believers”, and enough people to trust or follow them. They will (and have) return, and they will be (and are) stronger. If they don’t get in through the front door, they will use every crack in the wall to slowly infest any community from the fringes inwards. It’s just deferring a problem that was not created in the space of moderation and curation.

                                                                                      Again: This is not an argument for or against banning. I’m just saying nothing works, and that should be consciously realised.

                                                                                      1. 6

                                                                                        You just need a few “true believers”, and enough people to trust or follow them.

                                                                                        “Deplatforming” takes away the second part.

                                                                                        People who had huge followings on major social-media sites suddenly have far far smaller followings when kicked off, because they no longer have the major sites’ algorithmic “suggestion” systems giving them free promotion to millions or even billions of eyeballs. And that switch, from having new people passively funneled to you en masse by the original platform, to needing your existing audience to actively follow you somewhere else and actively promote you to people not already on the new platform, typically comes with a multiple-orders-of-magnitude drop in reach and following.

                                                                                        I believe that’s also in part why reddit’s “quarantine” feature exists; one effect of quarantining is that it yanks the subreddit out of automated promotion/suggestion by the site’s algorithms, which makes it far harder to recruit across the site through getting things splashed onto random users’ home-page views of reddit.

                                                                                        1. 2

                                                                                          People who had huge followings on major social-media sites suddenly have far far smaller followings

                                                                                          Sure, when it’s about individuals you’re concerned about then de-platform as much as you want to. But watch out, not that this one is gone, three others are trying to fill the hole he left behind. But seriously, a twitter account, a youtube channel or whatever is just an appearance. Anyone who used image boards knows how much even a small group of creating individuals, even if nobody ever finds out who they are, can do. The site can be shut down, but they can just as easily reconstitute themselves anywhere else. Maybe it takes a while, but just pushes people further.

                                                                                          1. 3

                                                                                            But watch out, not that this one is gone, three others are trying to fill the hole he left behind.

                                                                                            You say these things as if they’re just natural ways of the world, as if they’re true, but they’re just not.

                                                                                            When /r/fatpeoplehate was banned why didn’t /r/largepeopleanger and /r/hatethosebigfolks and /r/hatefats spring up in its absence? When Cloudflare deplatformed 8chan why didn’t 16chan and 32chan and 64chan immediately rise up from the ashes? When what’s-his-face who did all that heinous shit to the Sandy Hook parents was banned from all his vlogging and podcasting channels, why didn’t he and his fans just create dozens more?

                                                                                            When you de-platform someone or something that’s built a substantial audience, the creator and the audience have to do a lot of work to build themselves back up to their previous levels. And it’s a lot harder when the platforms that drive the highest engagement and acquisition numbers won’t host your shit anymore.

                                                                                            1. 3

                                                                                              When /r/fatpeoplehate was banned why didn’t /r/largepeopleanger and /r/hatethosebigfolks and /r/hatefats spring up in its absence?

                                                                                              As far as I remember, there were a few subreddits that came up to replace them, but all of them were shut down in their infancy. But then again, you’re confusing the forum for the people, they didn’t disappear. It’s internet pre-history by now, but it was one of the rallying calls leaving reddit, and was used as an example for how “SJW” are taking over. This lead to voat, 4chan exodi, and still is part of their impulse.

                                                                                              When Cloudflare deplatformed 8chan why didn’t 16chan and 32chan and 64chan immediately rise up from the ashes?

                                                                                              Oh there are millions of image boards that are trying to fill their absence, but that takes a bit. infinitychan also had to prove itself after all. But you’re right, until then, they are weakened. And if all you’re after is short term goals, good job. But again, the people, the ideas, the images are all still there, preparing to regather. And I’ll bet that this will incentivise more people than ever before to look into distributed alternatives, that will be harder to “de-platform”, because just like the users, they will have no platform they rely on.

                                                                                              When you de-platform someone or something that’s built a substantial audience, the creator and the audience have to do a lot of work to build themselves back up to their previous levels.

                                                                                              You’re doing it again. I’m not talking about individuals or “content creators”, they are worthless. It’s the same kind of thinking that leads people to believe that if Hitler were killed in WW1, there would’t have been any nazis. It’s an underestimation and fatally a lack of understanding what is being dealt with.

                                                                                              1. 2

                                                                                                I’m not talking about individuals or “content creators”, they are worthless.

                                                                                                What are you talking about, the ideas themselves? The movements?

                                                                                                Movements are only as strong as their adherents, the people behind them. Making it harder for the movements’ content creators to reach and engage audiences is nearly as good as somehow stopping people from being bigots in an abstract sense. It’s not the same but the net effect on a society is approximately equivalent. And more to the point, it’s one of the few ways that a society has traction in fighting these antisocial contagions.

                                                                                                1. 2

                                                                                                  What are you talking about, the ideas themselves? The movements?

                                                                                                  Am I really that incomprehensible? Is what I am saying that foreign? I’ll quote myself:

                                                                                                  banning people from reddit has made them relocate to voat, banning threads [made by… people] on 4chan has made them relocate on infinitychan

                                                                                                  The site can be shut down, but they can just as easily reconstitute themselves [ie. the community, of… people] anywhere else. Maybe it takes a while, but just pushes people further.

                                                                                                  But again, the people, the ideas, the images are all still there, preparing to regather

                                                                                                  The actually existing people behind these posts. Do you think the harassers and trolls aren’t driven by conviction? Do you think racists think what they do because they are bored? These “movements” are movements of “content creation”, not lead by them. Those parts of these committees that people are always talking about, would be the last to give up because of inconvenience. They literally think there is a world conspiracy against them. It’s just not that simple.

                                                                                                  It’s not the same but the net effect on a society is approximately equivalent.

                                                                                                  But again, t e m p o r a r i l y.

                                                                                                  1. 2

                                                                                                    Do you think the harassers and trolls aren’t driven by conviction?

                                                                                                    Yes, I think conviction merely provides the rationale for what they’re doing, I think the vast majority of their actual output is primarily driven by dopamine responses from audience engagement. And if you take that away, conviction alone won’t be enough for them to meaningfully continue. Not that it matters: if they want to bleat into the void and have nobody hear them, that is a complete victory from my perspective. I’m concerned about macro-scale effects on society.

                                                                                                    Do you think racists think what they do because they are bored?

                                                                                                    In our zeitgeist, on the internet platforms we’re currently talking about? Yes, actually. That’s a huge part of it. And things that are boredom-adjacent: a sense of community, dopamine from engagement, etc.

                                                                                                    It’s the 90/9/1 thing that applies to any online community, the fact that it’s about hate ideologies is irrelevant. 1% of the people are the true believers and actually producing content, 9% are highly engaged and curating/amplifying/whatever, but 90% are lurkers, consumers, a passive audience that is fickle and will disappear if you can deplatform the 10% from the most popular N sites on the internet.

                                                                                                    Every society will always have some bigoted assholes, and they’ll always have some kind of cult of personality or ideology that will attract some people. That’s unavoidable, those 10%. What’s avoidable is letting those subcultures attract and grow their 90% audiences. That’s the shit that tends to produce the lone-wolf spree shooters, tends to normalize microaggressions in day-to-day life, and most everything in between. And deplatforming is a really good tool for stopping that specific thing. Which is huge.

                                                                                                    1. 1

                                                                                                      In order for this to not go on forever, I’ll try to just summarise what I see our disagreements being:

                                                                                                      1. You’re concerned about the “macro-scale effects”, while I am more worried about the long-term effects.
                                                                                                      2. You think that racism is fuelled by boredom, while I think that is has deeper roots (although it can be set of by (life) boredom).
                                                                                                      3. You think that the 90/9/1 rule still applies, I think that the power/danger of the new communities comes exactly from transcending it.

                                                                                                      Unless you have anything else to contribute, I think it would be better to come to an end with this thread. The only question I have for you is what your direct experience is with these underground forms and image boards?

                                                                                4. 1

                                                                                  Coddling is a silly book rife with contradictions, it doesn’t strengthen your case to namecheck it https://www.theguardian.com/books/2018/sep/20/the-coddling-of-the-american-mind-review

                                                                                  1. 2

                                                                                    Guardian is a silly newspaper rife with contradictions. What’s your point?

                                                                                    1. 3

                                                                                      I have no real stake in the Guardian as a publication overall; it’s got a lot of crap, and some decent articles every now and then. I thought the review did a great job demonstrating why Coddling is a silly book. That should have been obvious.

                                                                                      By just attacking the least important part and not engaging with what I linked in any substantive way, I suspect you’re not arguing in good faith, saying quippy aggressive things. Come back if you want to address the contents of the review, or Coddling.

                                                                                      1. 1

                                                                                        I read this review and I’ve read the book. Frankly, the review doesn’t really address the book. This review is polarized and mentions some parts of the book totally out of context, and then tangentially starts talking about politics and Trump. The book is a really good read, and it’s well sourced. I looked up several of the stories it mentioned while reading it and I think it does a fair job of portraying what’s happening in a lot of universities, especially on the west and north east coasts.

                                                                                        There is a growing distrusted between those who teach in academia and their own students. Call-out culture is a a thing. There is a growing trend to react today first and to call for resignations and dismissals; to the point where I know people in academia who are afraid to talk about any difficult or hard issues.

                                                                                        It’s not a silly book. I’ve listened to other interviews with people like Haidt (one of the authors) along with people like Sam Harris who have brought up these same issues. Harris and Haidt is often labelled as alt-right or alt-right adjacent (same with Joe Rogan), but reading and listening to their views, they’re hardly that! And this goes back to the issue of calling everyone you don’t like or disagree with a Nazi or White Suprematist (especially those who don’t self-identify as such). It pushes more of this polarization narrative and people who have never even listened to these people now immediately dismiss everything they say.

                                                                                        1. 4

                                                                                          Thanks for actually engaging on this. I happen to disagree with almost everything you wrote (I read the book and think the review is dead-on) and think Sam Harris is a pompous Islamophobe and Joe Rogan is boring, mainstreaming people with terrible views by never challenging them, &c; &c;

                                                                                          I doubt we’ll get very far hashing it out here (and I have work to do lol, you probably have other responsibilities than hashing this shit out with a stranger on the Internet) but again, appreciate you rising up and responding sincerely 😄

                                                                                5. 3

                                                                                  I whole-heartedly believe that freedom of speech is an end in itself, but that doesn’t mean I’m against moderation. Not at all – too many communities are ruined by unpleasant, ill-willing people, and I think this very site is a clear example of how important content moderation can be.

                                                                                  Reddit, however, is a different type of site, one with many communities that are more or less separate from each other. Already before those hateful subreddits were banned, what they wrote in their walled garden never reached the eyes of redditors on the outside, unless they willingly looked inside – in this sense, there’s really no difference between subreddits and separate websites. So why were these subreddits banned? Well, because of pressure from other redditors, peeking inside the walled garden and not liking what they saw, and shareholders, presumably.

                                                                                  Of course, even though there’s little difference between having your community on Reddit versus hosting it on Voat, in the sense described above, Reddit is a bigger platform with more users than Voat. Being expelled from Reddit severely limits the user base of a community, which can be used both as an argument for banning these communities and as an argument for being careful about banning any communities whatsoever.

                                                                                  Anyway. I’m not defending these communities. My point is just that these bans weren’t really examples of content moderation, but rather, giving in to large amounts of criticism, valid or invalid as it may be.

                                                                                  1. 3

                                                                                    From your article:

                                                                                    Those with the power to do so have both the right and ethical obligation to stop these infections at the source, by organizing, by protesting, by de-platforming, and by recognizing that free speech isn’t an end in itself, but merely a means, a tool, which we’re obliged to use to make our society better.

                                                                                    I see nothing in that sentence with which Tourquemada would have disagreed.

                                                                                    We spent the last four hundred years building a world in which might doesn’t make right. We built a society which tries really hard not to unperson dissenters. It’s taken a long time, and it hasn’t been perfect, but we did it. And now, in just about a decade and a half, we have thrown away four centuries’ hard work and created a world in which it is once again no longer possible to speak truth to power, because once again those in power feel comfortable using that power to extinguish dissent and dissenters.

                                                                                    1. 1

                                                                                      We built a society which tries really hard not to unperson dissenters . . . And now . . . [we] created a world in which it is once again no longer possible to speak truth to power . . .

                                                                                      Look dude if literal Nazi-ism and white supremacy qualifies as “dissent” and “speaking truth to power” to you then we’re not going to be able to have a productive conversation. And to be extremely clear that is explicitly and only what this discussion is about. Not abstract and undefined “uncomfortable political ideas” or “unpopular opinions” or other weasel phrases. This whole conversation is about the alt-right race-baiting white-supremacist trolls of Voat.

                                                                                    2. 2

                                                                                      I think there is a large part of not being able to combat these ideologies because many people just don’t understand the fundamentals of them to begin with. That makes it easier to push people to extreme theories or ideologies because, in many cases, they put forth simple arguments for them or against whatever they’re against. It’s kind of like an ELI5 for ideas.

                                                                                      I’m not opposed to moderation and I think outright inflammatory posts should be removed immediately, but I also think we should be educating people better about ideas and not just resorting to calling people names. There are arguments against these ideologies, but we don’t have a general populace explaining, in an easy to digest way, why they aren’t good. I think we’re too quick to try to silence, which really doesn’t silence at all, but pushes the fringe folks together where their ideas echo and ultimately amplify.

                                                                                    1. 7

                                                                                      I don’t like DRM, but I don’t like this extremely dramatic doomsayer tone about DRM either. We’ve had DRM in general purpose computers for what feels like ages now, and nothing truly apocalyptic has happened. General purpose computing still exists. You still run free software. Millions of people still get their movies from The Pirate Bay. YouTube/Twitch/etc have not even considered using DRM to force people to watch ads together with the content. (They’re not even fighting youtube_dl really.) No one has tried to use EME for non-video content.

                                                                                      [Firefox adding EME support] did absolutely nothing to stop them from being steamrolled by Chrome’s growing popularity

                                                                                      uhh, how can we know that? We don’t exactly have an alternate reality where Firefox said no. We don’t know how many users would’ve quit Firefox specifically because it didn’t play Netflix.

                                                                                      1. 9

                                                                                        Adding EME to Firefox made more content publishers choose to enable EME since all (major) browsers supported it. Had Firefox not supported it, that decision would not have been so easy…

                                                                                        1. 8

                                                                                          NetFlix would not have backed down. They couldn’t, because they were competing for studio contracts with systems that relied on native apps (like iTunes, and their own offering on Android and iOS), systems that relied on plug-ins (like their browser-based player used to), and systems that relied on dedicated hardware (Blu-Ray, cable TV packages, and PlayStation). They could not possibly negotiate for a DRM-free contract when all of their competitors had DRM.

                                                                                          1. 3

                                                                                            My comment was specifically not about Netflix, but about smaller players… specifically tax payer paid for public TV in a least one country in Europe where that’s a thing. There was a brief and happy time between Flash, later Silverlight and EME where you could just point your browser to the site and watch the videos, even live TV! No plugin, no DRM. Of course as soon as EME became available, it was enabled. Hello infinite spinner not loading the video! :)

                                                                                          2. 5

                                                                                            And the pressure on Firefox would not have been there, had the W3C not betrayed web users.

                                                                                            1. 5

                                                                                              True. probably, maybe. The W3C is corrupt. See e.g. https://ar.al/notes/we-didnt-lose-control-it-was-stolen/, or the story about EFF’s withdrawal from W3C at https://www.eff.org/deeplinks/2017/09/open-letter-w3c-director-ceo-team-and-membership.

                                                                                        1. 20

                                                                                          Am signatory, AMA.

                                                                                          1. 5

                                                                                            Were there any project leaders that refused to sign?

                                                                                            1. 6

                                                                                              Let’s provide some context here, shall we?

                                                                                              There’s been 20 signatories, and one of them isn’t even a maintainer of any package (they’re a staff member).

                                                                                              There’s close to 400 GNU packages, plus close to 100 additional discontinued GNU packages:

                                                                                              E.g., about 5% of folks singed this. Many bigger packages like GCC would have more than one maintainer, too.

                                                                                              Additionally, it’s been pointed out on another platform that this whole thing is a Guix’ response to disagreeing with Dr RMS on his GNU Kind Communications Guidelines some 11 months ago, because they weren’t punitive enough:

                                                                                              I’d say the whole thing was brewing for quite a while. Would be surprised for the list of signatories to change in any significant manner. Just looking at these numbers and the dates, I’d be surprised if many more folks haven’t been afforded the opportunity to join the mob, but didn’t. The fact that they hide all these things reveals their methods of action.

                                                                                              1. 5

                                                                                                We are not hiding anything. Stallman is not a victim. We are not a mob. We are a collective of GNU maintainers who have had enough, and we’re hardly alone in the world with having had enough with RMS. He’s had good philosophies that persuaded all of us at one point, but his leadership and communication have been sorely lacking.

                                                                                                I actually expect the number of signatories to increase a little. I know of at least a few who wanted to sign but just didn’t get around to it because they were busy. Of those 400 GNU maintainers, most are inactive. GNU is not as cohesive as you might think, which again I think shows lack of good leadership.

                                                                                                Yes, there’s only 20 or so of us, but we represent some of the biggest GNU packages.

                                                                                                1. 1

                                                                                                  We are not hiding anything. Stallman is not a victim. We are not a mob. We are a collective of GNU maintainers who have had enough, and we’re hardly alone in the world with having had enough with RMS. He’s had good philosophies that persuaded all of us at one point, but his leadership and communication have been sorely lacking.

                                                                                                  I actually expect the number of signatories to increase a little. I know of at least a few who wanted to sign but just didn’t get around to it because they were busy. Of those 400 GNU maintainers, most are inactive. GNU is not as cohesive as you might think, which again I think shows lack of good leadership.

                                                                                                  Yes, there’s only 20 or so of us, but we represent some of the biggest GNU packages.

                                                                                                  There’s so much misrepresentation here I don’t even know where to begin.

                                                                                                  There’s already at least a couple of people on the list that aren’t even developers.

                                                                                                  You refer to yourself and all other signatories as “GNU maintainers”, including the “GNU Octave maintainer” on your hat, but what does it mean exactly?

                                                                                                  Not familiar with GNU Octave, I originally got the impression that you were the sole person responsible for the project. In fact, that’s what the word “maintainer” means in most other projects. Which, per further examination, cannot be further from the truth — there’s a bunch of commits over at http://hg.savannah.gnu.org/hgweb/octave, and none of them seem from you. When searching for your name, http://hg.savannah.gnu.org/hgweb/octave/log?rev=Jordi, we get a whole 10 results, spanning 2014 to 2017. Do you use some other ID within the project? Or is this pretty much representative of your involvement with the project you claim to be an official representative of? Wikipedia has a link to http://hg.savannah.gnu.org/hgweb/octave/file/tip/doc/interpreter/contributors.in, which reveals that there are a whole of 445 contributors to GNU Octave, and you’re the only one of these people who is a Guix signatory listing Octave.

                                                                                                  Sure, some of the folks on the list are actual maintainers and/or are responsible for significant work. But do you even fail to see how simply putting a random list of semi-active part-time and drive-by developers as signatories behind cancelling the founder and 80-hours-per-week full-time advocate of Free Software is not exactly representing things as they are? How’s that not a mob?

                                                                                                  Also, what is your exact intention when presenting yourself and everyone else as a “maintainer”, and with statements like “we represent some of the biggest GNU packages”? Were you officially designated to speak on behalf of any of these projects? Or is the whole intention to confuse others in a way similar to how you had me confused with your hat here on Lobste.rs? I don’t have time to check out every name (and some do checkout, some don’t), but it is beyond obvious that you don’t actually represent the views of GNU Octave as you imply, and presenting yourself as an active “maintainer” shows that you have no interest in spreading any truths anywhere, either.

                                                                                                  1. 5

                                                                                                    As much as I dislike the backstabbing of this “joint statement” by GNU developers, I have to say that you are grossly mis-representing JordiGH contribution to Octave. He’s easily the main scientific contributor to this project after Eaton himself (which makes me even sadder that he’s actually signed the backstabbing manifesto).

                                                                                                    1. 2

                                                                                                      He’s been busy, but jwe finally got around to signing it too. 24 signatories now.

                                                                                                      1. 3

                                                                                                        I’m very sad to hear about that. From the outside it looks like you are part of the pithy smearing campaign against free software. I fail to understand how this “joint statement” at this moment helps anybody (besides mattl and the like).

                                                                                                        I admire the work of most people who signed this statement, and jwe is one of my heros and sources of inspiration–as much as RMS. Even if I agree with the principle that the FSF/GNU leadership can change for the good, the second part of the statement that you signed reads as a callous backstabbing. I literally cried when I read the list of signatories. I cannot help but feel a bit guilty today when recommending octave to my students.

                                                                                                        1. 1

                                                                                                          GNU leadership and its structure needs to change. Hell, GNU needs a structure to begin with – we don’t have any sort of organisation yet and thus our ties and cohesion between GNU packages over the years have weakened.

                                                                                                          Even if RMS were a perfect saint and the hero many of us made him out to be, nobody should be appointed leader for life. We rotate other leadership positions, and we should do the same with this one.

                                                                                                          1. 4

                                                                                                            I agree 100% with what you say here, but not with the public statement that you signed, which alienates me.

                                                                                                    2. 1

                                                                                                      He’s been busy, but jwe finally got around to signing it too. 24 signatories now.

                                                                                                  2.  

                                                                                                    Who is the staff member?

                                                                                                  3. 3

                                                                                                    I don’t know. I wasn’t the one doing the outreaching.

                                                                                                  4. 4

                                                                                                    How was this coordinated?

                                                                                                    1. 8

                                                                                                      Private emails. We all were kind of aware of each other and Ludovic started an email thread where we discussed this.

                                                                                                    2. 4

                                                                                                      You all planning to replace RMS with a new “chief GNUsciance”, or planning to switch to a steering council like Python did?

                                                                                                      If there is no plan, then which one do you prefer?

                                                                                                      1. 9

                                                                                                        No plan yet, just a plan to discuss. I am personally in favour of a steering committee. It seems to have mostly worked for gcc. I got to see some gcc people a couple of weeks ago for GNU cauldron, and that was fun. I would like something more like that.

                                                                                                      2. 2

                                                                                                        I’m confused by this FSF statement: https://www.fsf.org/news/fsf-and-gnu.

                                                                                                        It links using “GNU leadership has also published a statement”, which kinda implies with the surrounding text that GNU leadership is multiple people, but the link target is mail by Stallman saying that he will talk to FSF as a single person.

                                                                                                        https://lists.gnu.org/archive/html/info-gnu/2019-10/msg00004.html

                                                                                                        Is there anyone else or is this just a language oddity?

                                                                                                        1. 3

                                                                                                          Just a language oddity. As of right now, nothing has changed and “GNU leadership” is synonymous with “RMS”.

                                                                                                        2. 2

                                                                                                          So, if rms resigns from GNU and suffers any negative mental health outcomes, would you believe yourselves to be contributing factors or perhaps even responsible?

                                                                                                          1. 27

                                                                                                            Let’s not play into “if you leave me, I’ll hurt myself and it’ll be your fault” abuser playbook.

                                                                                                            RMS should get help if he needs it, but not in the form of coddling him in a position of power he’s unfit for.

                                                                                                            1. 9

                                                                                                              I don’t know about abuser playbooks, I’m just thinking about it in terms of common decency for folks that have had internet mobs arrayed against them (correctly or incorrectly).

                                                                                                              I certainly think it would be tacky if, say, a bunch of trolls got somebody ousted from their position in an open-source project and then refused to take responsibility if that person was harmed. The only salient difference to me here seems that you think (and correct me if I’m wrong!) of rms as an acceptable target.

                                                                                                              1. 10

                                                                                                                RMS getting fired over the Minsky remarks is utter bullshit, and it was a total violation of due process, journalistic integrity, and other niceties of civilization… but that doesn’t mean he should be in a leadership position. I think the the whole Epstein business was used as a pretext for people who already wanted him out (for good reasons) to kick him out (based on a bad reason).

                                                                                                                Which is to say, it’s not entirely that simple.

                                                                                                                1. 3

                                                                                                                  RMS getting fired over the Minsky remarks is utter bullshit,

                                                                                                                  He wasn’t fired. He voluntarily left of his own accord, because of comments that he made, while interjecting into a conversation that he was not originally part of. The comments are in line with culturally taboo statements he has made public on his website for over 20 years that people have willfully ignored for the sole reason of giving him the benefit of the doubt. This time, he crossed a line because a) the statements that he made are incredibly adjacent to, and almost identical to, arguments made by people who abuse young children (Regardless of his intent) and b) there were abuse survivors in the conversation that he interjected into, that were likely affected by those statements.

                                                                                                                  and it was a total violation of due process, journalistic integrity, and other niceties of civilization…

                                                                                                                  Well, no. Not only is his position as chairman not subject to those concerns, he himself violated said niceties of civilization.

                                                                                                                  but that doesn’t mean he should be in a leadership position. I think the the whole Epstein business was used as a pretext for people who already wanted him out (for good reasons) to kick him out (based on a bad reason).

                                                                                                                  Indeed. The word is that he has continually scuppered several projects (Including GNU’s version of DotNET which had a presence on the steering committee!!!) which caused non-GNU alternatives to have the upper hand, defeating GNU’s objectives of software freedom in the process.

                                                                                                                  1. 8

                                                                                                                    Pretending his exit was voluntary is disingenuous.

                                                                                                                    1. 4

                                                                                                                      he himself violated said niceties of civilization.

                                                                                                                      One of the niceties of civilization is the rule of law, in particular “just because you broke the rules doesn’t mean I get to”. So that’s irrelevant.

                                                                                                                    2. 0

                                                                                                                      They railroaded a guilty man, in other words?

                                                                                                                      1. 3

                                                                                                                        Not sure I follow the phrasing, but perhaps “a good thing done badly” might describe it, depending on whose stories you give credence to.

                                                                                                                    3. 7

                                                                                                                      Part of leadership is your subordinates not wanting to be lead by you anymore. This doesn’t make him a target.

                                                                                                                      Harm reduction may be a goal in these situations and, if you have a look at the statement, it gives appropriate credit to RMS, but also makes it clear that his time is over.

                                                                                                                  2. 17

                                                                                                                    He’s fine. We’re not responsible for his behaviour or his health. He is, and his own actions over the decades are.

                                                                                                                    But really, he’ll be fine. He’s not a martyr. We need a change in leadership and he needs time to reflect.

                                                                                                                  3. 2

                                                                                                                    What’s the big deal?

                                                                                                                    1. 8

                                                                                                                      I don’t understand the question. Big deal about what?

                                                                                                                    2. 1

                                                                                                                      Perhaps I’m out of the loop. I’m aware of Stallman’s anti-social behavior in the past, but is there some new reason this is happening now, rather than years ago?

                                                                                                                      Edit: Oh, I am definitely out of the loop. I just read about Stallman’s Epstein remarks. How vile.

                                                                                                                      1. 10

                                                                                                                        If you ask me (which I think you did), this should have happened years ago, but yes, the recent incidents were the final push we all needed.

                                                                                                                        1. 2

                                                                                                                          I don’t think that the Epstein remarks, at least what I’ve heard of them, are anything new or surprising if you’ve followed Stallman for a while. It’s not out of character at all.

                                                                                                                        2. 1

                                                                                                                          Well, it may be nice to have a different leadership for the GNU project. Why not discuss it with the man himself? Has anyone tried before going public?

                                                                                                                          1. 1

                                                                                                                            We’re trying to discuss different leadership. And they’re trying to not go public. I don’t think i can say much more without being unkind.

                                                                                                                            1. 1

                                                                                                                              So I guess, that’s a no. “Unkind” is too kind a word.

                                                                                                                              Edit: to clarify this comment, this all reeks of “the ends justify the means”. While I agree with the ends, the means do not look good, and it changed how I perceive both RMS & the projects under the GNU umbrella.

                                                                                                                              I hope I did not sound angry. I’m just annoyed at myself (mostly). I wish you luck in this endeavour and other future projects. :)

                                                                                                                        1. 43

                                                                                                                          I would guess that a very substantial proportion of the people who read Lobsters have heard of Timsort.

                                                                                                                          1. 22

                                                                                                                            That, and TimSort really only makes sense in certain situations:

                                                                                                                            • Comparisons have to be extremely expensive. Think dynamic dispatch. This is because TimSort itself performs a bunch of branches and integer comparisons to keep track of galloping scores and the stack invariant.

                                                                                                                            • You need a stable sort. If you don’t need a stable sort, pattern defeating quicksort will probably do better.

                                                                                                                            1. 11

                                                                                                                              Quicksort is the Achilles of sorting algorithms: unbeatably fast, easy to implement, in-place; but with the vulnerable heel of bad worst-case performance (the worst case being pre-sorted data in the naïve implementation) and instability.

                                                                                                                              1. 5

                                                                                                                                There’s a fairly easy fix to that, called introsort: start with quicksort, but bail out to a guaranteed O(n log n) sort like heapsort if it takes too long. In the bail-out case, you lose constant-factor performance compared to if you had used heapsort in the first place, but you avoid quicksort’s O(n^2) worst case, while still getting its good performance in non-pathological cases. It’s used in practice in .NET and some C++ STL implementations.

                                                                                                                                1. 3

                                                                                                                                  Quicksort -> Heapsort is method I used. It worked fine in practice. I love solutions like that. Another, unrelated one was sklogic’s trick of using a fast, dumb parser first to see if it’s correct. If it wasn’t, he switched to one that made error messages easier.

                                                                                                                                  I bet there’s more of this stuff waiting to be found for situations where people are shoving every case into one algorithm.

                                                                                                                              2. 4

                                                                                                                                Comparisons have to be extremely expensive. Think dynamic dispatch.

                                                                                                                                That explains why Python uses it as it’s standard sort.

                                                                                                                                1. 9

                                                                                                                                  Yeah. That’s exactly why Python uses TimSort.

                                                                                                                                  More tellingly, where Rust uses an algorithm that’s related to TimSort for its stable sorting algorithm, they didn’t implement “galloping” because it’s not worth it. https://github.com/rust-lang/rust/blob/7130fc54e05e247f93c7ecc2d10f56b314c97831/src/liballoc/slice.rs#L917

                                                                                                                              3. 10

                                                                                                                                I consider myself relatively knowledgeable about many different topics of CS and had not heard of Timsort until this article. What’s the point of your comment? That the article is not worth posting as you presume that it is widely known?

                                                                                                                                1. 4

                                                                                                                                  The point is that the title of the article, and of this submission, is inaccurate. I would call the title clickbait because for most readers, the article doesn’t deliver what it promises – a sorting algorithm “you’ve” never heard of. I think the article itself is fine; it’s just the title that is a lie.

                                                                                                                                  1. 5

                                                                                                                                    That seems to be a really low-value comment. For whom is the remark actually intended? For other crustaceans to look at, nod and agree, thinking, “yes, I too possess the superior knowledge”? Does every submission with a title that invokes “you” need to be correct and ‘deliver on its promise’ for all possible “you”s? C’mon.

                                                                                                                                    1. 4

                                                                                                                                      Yes, I suppose jgb could have been more explicit in why they brought up their guess. (I wrote an explanation of my interpretation of the comment, not the original comment.)

                                                                                                                                      Does every submission with a title that invokes “you” need to be correct and ‘deliver on its promise’ for all possible “you”s?

                                                                                                                                      I think every article with a title that invokes “you” needs to be correct and ‘deliver on its promise’ for the majority of possible “you”s in its audience. If a title says “you’ll love this” and most readers don’t love it, the title was wrong, and it wasted people’s time by getting them to open the article on false pretenses. It is up to article authors to adhere to that principle or not.

                                                                                                                                      As for titles of submissions of articles with clickbait titles, there can be a conflict between submission titles that reflect the author’s intent and titles that accurately describe the article. I don’t have a simple answer as to when submission titles should differ from the article title.

                                                                                                                                      1. 3

                                                                                                                                        I think every article with a title that invokes “you” needs to be correct and ‘deliver on its promise’ for the majority of possible “you”s in its audience.

                                                                                                                                        I think I agree with this, and I think my concern comes down to disagreeing instead with the notion that the majority(/“a very substantial proportion”) of Lobsters readers have heard of Timsort. Short of a poll there’s not an actual answer to that; I just felt particularly rankled because I hadn’t, and presumably if I had I wouldn’t have bothered or thought to comment myself.

                                                                                                                                        I err on the side of preserving the article title in the submission, which I think is pretty common. Accordingly, I think most Lobsters are primed to see submission titles that aren’t necessarily addressing them as Lobsters readers, but in some context that might be quite removed.

                                                                                                                                2. 2

                                                                                                                                  I thought it played a pretty big role in the Oracle vs. Google lawsuit too, making it one of the more famous algorihtms.

                                                                                                                                  However see “rangeCheck” mentioned a lot, which is a trivial part of TimSort.

                                                                                                                                  https://en.wikipedia.org/wiki/Oracle_America,_Inc._v._Google,_Inc.

                                                                                                                                  Here it seems to cite TimSort. But for some reason I can’t find a lot of sources that talk about TimSort and the lawsuit, even though at the time I remember it being a prominent thing.

                                                                                                                                  https://forums.appleinsider.com/discussion/149435/google-engineers-defend-source-code-email-in-oracle-lawsuit-over-java/p4


                                                                                                                                  edit: here’s another one mentioning TimSort and the lawsuit.

                                                                                                                                  https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/

                                                                                                                                  Googling “rangeCheck timsort lawsuit” digs up some results.

                                                                                                                                1. 14

                                                                                                                                  Cloudflare’s competitors rely on a DNS extension that provides some fairly coarse geographical information about clients upstream (the client subnet) in order to provide feedback about the geographical distribution of requests. Cloudflare blocks this with 1.1.1.1 claiming it’s for privacy reasons but that’s just using trigger-talk to justify an anti-competitive practice IMO. The archive.is folks are refusing to support non-EDNS resolvers.

                                                                                                                                  I’ve noticed a lot of breakage on the Russian internet when using 1.1.1.1 as well.

                                                                                                                                  In the early days, implementors of this DNS extension would sometimes use a manually configured whitelist to determine who they sent the augmented (and invalid-to-non-supporters) responses to, but it seems like now there are a number of resolvers that just send it to any requesting resolver.

                                                                                                                                  1. 3

                                                                                                                                    Cloudflare’s competitors rely on a DNS extension that provides some fairly coarse geographical information about clients upstream (the client subnet) in order to provide feedback about the geographical distribution of requests.

                                                                                                                                    That’s not entirely true. Archive Today relies on it, and I know they aren’t the only ones, but many of CloudFlare’s actual competitors don’t rely on EDNS.

                                                                                                                                    CloudFlare, Fastly, and Google all use BGP anycast for geographical distribution of requests. Basically, it’s a deliberate IP address conflict, but since each node is placed in a different location, the “shortest, fastest route” BGP logic does the load balancing. While AWS does use DNS to perform “latency-based” load balancing, they don’t rely on EDNS either: they can guess where the resolver is, geographically, based on which of their data centers wound up getting the request (AWS has lots and lots of peering arrangements, all of which lead to their handful of “region” data centers).

                                                                                                                                    If someone tried to bring an anti-competitive charge against CloudFlare for this, that’s what their reply would be. Their competitors don’t need EDNS, and neither do they. The websites that are actually impacted by this are ones that are big enough to benefit from a CDN, too small to make BGP peering arrangements, and too stubborn to make a deal with a company for their CDN.

                                                                                                                                    Killing off EDNS would help all of the big CDN providers, not just CloudFlare. They’re not hurting their competitors, but they are growing the market that they’re in.

                                                                                                                                    1. 4

                                                                                                                                      Just because a given provider runs an anycast network doesn’t mean that they don’t also use EDNS-Client-Subnet, too. If it was as useless as you claim, it wouldn’t have seen a rather good adoption.

                                                                                                                                      In fact, if you look at the responses from Cloudflare CEO, you can see that the big players actually do use ECS, and Cloudflare’s lower number of PoPs does seem to affect them:

                                                                                                                                      https://news.ycombinator.com/item?id=19828702

                                                                                                                                      We are working with the small number of networks with a higher network/ISP density than Cloudflare (e.g., Netflix, Facebook, Google/YouTube) to come up with an EDNS IP Subnet alternative that gets them the information they need for geolocation targeting without risking user privacy and security. Those conversations have been productive and are ongoing. If archive.is has suggestions along these lines, we’d be happy to consider them.

                                                                                                                                      Keep in mind that these are merely the bigger players, and the smaller players, other than archive.today, probably don’t even get a say in the matter at all.


                                                                                                                                      too stubborn to make a deal with a company for their CDN

                                                                                                                                      Yeah… No. The logic that everyone should be beyond one of the big CDNs is beyond flawed. Are you seriously blaming a hobbyist project, bootstrapped startup like archive.today, for not wanting to have corporate overloads like Cloudflare dictate what they can and cannot do for their hobby, and have complete control over the viability of the project? To have a dependency on opaque third-party SaaS? Describing this as stubborness is insincere at best.

                                                                                                                                      1. 4

                                                                                                                                        Stubborn isn’t necessarily bad. I apologize for using language that had such a negative connotation. Archive Today is not doing anything wrong, and it was bad for me to make it sound like they were. In the future, I’ll avoid using language that comes across so negatively.

                                                                                                                                        1. 1

                                                                                                                                          One of the biggest character flaws I see among engineers is a myopic focus on “just making it work”, regardless of “If you should make it work”. I feel that has appeared here, where small players must cater to big players.

                                                                                                                                          Granted, everyone loves the “big players shouldn’t cater to every small player” mindset, but it’s inherently reductive and leaves us at the mercy of the big players who can control everything.

                                                                                                                                          Perhaps EDNS is no longer necessary. However, I really think a form of “Be conservative in what you send, be liberal in what you accept” should apply to Big players most of all.

                                                                                                                                    1. 6

                                                                                                                                      EDNS is important. I don’t know how Cloudflare thinks you’ll get a good experience without it. It will cause odd behavior like an American being sent to an Australian CDN mirror.

                                                                                                                                      I was having this issue at work and then realized it was broken EDNS…

                                                                                                                                      1. 5

                                                                                                                                        I don’t know how Cloudflare thinks you’ll get a good experience without it.

                                                                                                                                        You get a good experience to Cloudflare hosted proxies. Which strenghtens their case when they say that your website gets faster if you let them MITM it.

                                                                                                                                        1. 2

                                                                                                                                          Don’t most global DNS/CDN provider rely on anycast + bgp for routing requests to closest PoP?

                                                                                                                                          1. 1

                                                                                                                                            That’s exactly the thing, isn’t it. CloudFlare, and many of their direct competitors like Fastly, do use anycast routing for their CDN. But that’s expensive, and making it work well is complicated, so Archive Today uses Geo DNS instead.

                                                                                                                                            1. 1

                                                                                                                                              While anycast is complicated it can be done cheaply by those who know how, and for those who don’t, route53 is only marginally more expensive than a well-monitored “geo dns” setup.

                                                                                                                                              “Geo dns” is so dumb, and so extremely error prone, and so very difficult to correctly handle many failure scenarios that are just simply automatic with the anycast approach, that I do not recommend it to anyone.

                                                                                                                                              1. 3

                                                                                                                                                can be done cheaply by those who know how

                                                                                                                                                OK, I’m all ears here!

                                                                                                                                                Anycast is completely inaccessible to the average advanced user. There are some providers which offer anycast service on their own IP space (vr.org/hostvirtual.com — only one provider I’m aware of, showing that there’s not even much competition), and you quickly realise just how inflexible the whole thing is, because IPv4 space has a /24 granularity, meaning, you gotta dedicate a whole /24 to a given anycast group, and every single member of the group is then bound to have anycast presence in every single location from which the anycast is announced.

                                                                                                                                                Got a new PoP? Gotta have every customer onboard for purchasing the computing resources at the new location. Got a spiked load for one specific PoP? Better be using autoscaling! I.e., it’s an all-or-nothing kind of situation. There’s no individual scaling, individual selection of which PoPs you get to have your anycast in etc.

                                                                                                                                                Otherwise, if you do want such control, you gotta have your own /24, and find providers willing to announce it for you. Where exactly can you get such custom services cheaply? I don’t see anything like that in the price lists of most hosting providers which are my go-to for the affordable dedicated servers.


                                                                                                                                                This whole notion and all the arguments provided by Cloudflare and their fans resolve around the fact that they simply don’t care about the hobbyist and grass-roots internet services like archive.today. The writing on the wall being that archive.today is too small to need their own thing, should be a (paying) Cloudflare CDN customer, or just do things differently than is convenient for them in their free service without venture capital, as well as other bigotry justifying why Cloudflare, a 3.5 billion US dollar company playing dirty with their free Cloudflare DNS subsidised by their billion-dollar CDN operation to slow down all the other competing CDNs, is in the right. It’s sad that so many folks applaud these actions — they should not be celebrated. We need internet diversity. Some folks don’t even realise how much of a monopoly and a bully Cloudflare already is. It is hard but possible to do internet without Google. Not so easily without access to Cloudflare.

                                                                                                                                                1. -1

                                                                                                                                                  Anycast is completely inaccessible to the average advanced user.

                                                                                                                                                  Be that as it may, I also think you’re probably less advanced than you think.

                                                                                                                                                  First issue: BGP has no security. Getting an IP and an ASN is just some paperwork, but you still need networks to let you announce.

                                                                                                                                                  I don’t know of any providers that would let you do this with a prepaid debit card you picked up in Walgreens, but once I’ve explained what I’m doing, I’ve found most virtual hosting providers will set things up appropriately. I’ve yet to have someone charge me for this.

                                                                                                                                                  Some of the providers I’ve used include Softlayer, Ghandi, HopOne, and OVH. Just one Linux instance, and I run gated (or zebra or whatever) on each site.

                                                                                                                                                  If you still can’t figure it out, you can suggest some use cases and maybe I would help you set up an experiment if you have an interesting idea.

                                                                                                                                                  There are a lot of things to get right if you don’t want to piss off other network admins on the Internet so it should not surprise you (or anyone else) that there’s a significant amount of KYC at the gates.

                                                                                                                                                  This whole notion and all the arguments provided by Cloudflare and their fans resolve around the fact that they simply don’t care about the hobbyist and grass-roots internet services like archive.today.

                                                                                                                                                  This is uncalled for.

                                                                                                                                                  I’m happy to engage on EDNS (client subnet extension) being a dumb/unnecessary privacy leak, and useless for load balancing or DDoS protection, and anycast being straightforward, but I can’t speak for Cloudflare or its fans. I’ve never used Cloudflare and my experience as a regular internet user would never have me recommend them or their patently silly “DDoS protection” product.

                                                                                                                                          2. -3

                                                                                                                                            It shouldn’t.

                                                                                                                                            If you have anycast dns servers who receive a request from a name server without EDNS support, you can basically assume the POPs nearest your nameserver should be returned.

                                                                                                                                            If you’re not anycast (eg because you’re the sad sort who thinks “Dns geoip load balancing” is a thing) then you should just fix that problem. Route53 is cheap.

                                                                                                                                            1. 3

                                                                                                                                              Yeah, sorry, maybe with all your millions of dollars in venture capital you might think that DNS geoip is not a thing, but if you don’t have that kind of money, it still does the job just fine. (Extra 80ms in latency is nothing compared to the average page load times of several seconds on the modern web anyways.) And EDNS-Client-Subnet isn’t there for nothing, either. And what does AWS Route53 has to do with anycast in the first place? Also — AWS may be convenient, but it sure ain’t cheap, either.

                                                                                                                                              1. 0

                                                                                                                                                with all your millions of dollars in venture capital

                                                                                                                                                Who the fuck do you think you’re talking to?

                                                                                                                                                I don’t have millions of dollars in venture capital.

                                                                                                                                                you might think that DNS geoip is not a thing

                                                                                                                                                Not a thing? It’s something that inexperienced sysadmin do: It’s clearly a thing, just not a good thing.

                                                                                                                                                And EDNS-Client-Subnet isn’t there for nothing, either.

                                                                                                                                                Yes. It really is.

                                                                                                                                                It’s a privacy leak engineered by Google, and there’s no point to it. You can’t trust it, so you must handle fallbacks with it missing instead of diverting traffic to 127/8 e.g. users who have a local resolver running on their laptop which might not know the IP address (or guess wrong). Just assume the web service closest to your nameserver and you’ll immediately be doing better.

                                                                                                                                                The users who foolishly use 8.8.8.8 and etc because they believe it’s faster deserve what they get. They are great tools, and it’s touching how they’re used to subvert censorship and oppression but make no mistake: the only real reasons they serve is those to their masters.

                                                                                                                                                AWS may be convenient, but it sure ain’t cheap, either.

                                                                                                                                                Route53 is cheaper than trying to build your own reliable “geoip dns” crap: one of the biggest cost-savings is you don’t need a few dozen monitoring nodes second guessing the answers coming out of your servers. Split paths and partial network outages are extremely common and impossible for your dns servers to detect on their own.

                                                                                                                                              2. 1

                                                                                                                                                We use GeoDNS and anycast. It’s not rocket science and there are some very good reasons to leverage both.

                                                                                                                                                1. 0

                                                                                                                                                  There aren’t. Really.

                                                                                                                                                  BGP is a much better/more-complete picture of the network shape, so there’s no reason to do an IP to arbitrary country or lat/lon table built every month by Ip2location (or your favourite vendor here), and given that database is obsolete before you even downloaded it, you might as well “just” use BGP and your health/monitoring network.

                                                                                                                                            1. 9

                                                                                                                                              Dumb polling makes a lot of sense - most people will not linger looking at those scores. Adding a bunch of extra code to the google home page is probably a pretty expensive thing to do at that scale. Using a stateful protocol for this would require a ton of servers.

                                                                                                                                              Sometimes technology from the 90s is the best answer 😃

                                                                                                                                              1. 1

                                                                                                                                                Hi Orestis. Matt here, author of that article. Thanks for the feedback. When you say adding a stateful protocol, I am confused. What is wrong with long polling, SSE or raw Websockets? Those can all be treated as stateless.

                                                                                                                                                1. 7

                                                                                                                                                  All of those keep state in the TCP and TLS implementations on the server. It’s not a lot per connection, but it adds up.

                                                                                                                                                  1. 1

                                                                                                                                                    Sure, agreed, there is definitely some state. Saying that, browsers will maintain HTTPS connections anyway, so there is some state maintained regardless.

                                                                                                                                                  2. 3

                                                                                                                                                    Apart from the TCP state, doesn’t the server need to keep state of which clients are connected, subscribers to some channel, logic to send new results to them, dropping clients who disconnect, dealing with fan-out when you get more connections than a single machine can handle, etc?

                                                                                                                                                    Whereas this http only thing is just dumb data that could sit on any caching layer in between, can benefit from a whole bunch of http semantics for caching.

                                                                                                                                                    If you want sub-second precision of tons of data I get that web sockets or some other server push tech might be a better fit - but not in these case, in my opinion.

                                                                                                                                                    1. 2

                                                                                                                                                      More importantly, while HTTP persistent connections are certainly terminated at Google’s level 7 load balancing tier, they probably terminate push sessions at the application. Terminating a websocket connection in the load balancer would only make sense if the load balancer performed fan-out, which would benefit this use case, but would not benefit use cases like GMail where every user gets a disjoint set of messages anyway.

                                                                                                                                                1. 4

                                                                                                                                                  Package manager written in POSIX sh

                                                                                                                                                  busybox ash

                                                                                                                                                  shellcheck

                                                                                                                                                  I see a problem here. Ash doesn’t support arrays (and shellcheck will enforce that word splitting is not used as a substitute). While that may be adequate in some special cases, it’s a straitjacket I generally wouldn’t want to be programming in.

                                                                                                                                                  The reason I wrote a guide called how to do things safely in bash specifically, and not in POSIX shell in general, is that there are things (like common things) that just can’t be done correctly in POSIX shell.

                                                                                                                                                  1. 2

                                                                                                                                                    I see a problem here. Ash doesn’t support arrays (and shellcheck will enforce that word splitting is not used as a substitute).

                                                                                                                                                    Positional parameters are used for arrays; ShellCheck’s warnings are disabled temporarily when intended. Think of it like Rust’s unsafe: you can use it safely, you just have to be careful as you’re on your own there. Globbing is, of course, disabled when this is done.

                                                                                                                                                    The package system’s design is also compatible with POSIX shell: each type of entry has a separate file (e.g., checksums) and has one entry per line. So it can be easily looped over by redirecting it into a while read loop.

                                                                                                                                                    The reason I wrote a guide called how to do things safely in bash specifically, and not in POSIX shell in general, is that there are things (like common things) that just can’t be done correctly in POSIX shell.

                                                                                                                                                    Yes, the author is experienced in Bash; he’s the author of Neofetch, Pure Bash Bible, fff, etc.

                                                                                                                                                    The final blow is that fighting such an abstraction failure of the language is pointless if you can choose a different language.

                                                                                                                                                    POSIX shell has advantages, though; in this case, it’s minimal and part of BusyBox. Bash is completely incompatible with the base system’s philosophy with its bloat, however great it might be, so you can’t simply choose it here.

                                                                                                                                                    1. 2

                                                                                                                                                      Solid answer. Thanks.

                                                                                                                                                      I still think that someone needs to fork Busybox/Ash and add array support. It probably wouldn’t add much bloat (since it already supports one array – the positional parameters, as you mentioned).

                                                                                                                                                      That would remove this unnecessary burden on the responsible programmer to select $IFS carefully (which is not generally possible!), turn off globbing and reason about safety on their own.

                                                                                                                                                      And with that goes (AFAIK) the only unachievable aspect of shellcheck compliance. Ash would move from bad™ to good™ on my list.

                                                                                                                                                      1. 2

                                                                                                                                                        KISS is designed to be minimal and substitutable, though, so the package manager is going for full POSIX compliance, and it seems unlikely for POSIX to implement that. The author did mention that arrays were the main thing he missed from Bash.

                                                                                                                                                      2. 2

                                                                                                                                                        Think of it like Rust’s unsafe: you can use it safely, you just have to be careful as you’re on your own there.

                                                                                                                                                        Safe Rust has robust support for iteration, though. It isn’t utterly crippled.

                                                                                                                                                        1. 1

                                                                                                                                                          Yes, safe Rust is much more powerful overall than POSIX sh; that’s not the point of my analogy.

                                                                                                                                                    1. 12

                                                                                                                                                      The decision to not include gettext and intltool highlights the major problem with this “simplicity” obsession which is present in several subcultures in the computing community.

                                                                                                                                                      Of course simplicity is a good thing to aim for, but some problems we have to solve are complicated. If your philosophy can provide a simple way to solve those problems then fantastic, but if it merely refuses to solve them at all then I think it might be time to ask questions about the worldview you’re working with.

                                                                                                                                                      1. 6

                                                                                                                                                        gettext and intltool

                                                                                                                                                        Are they really needed on servers?

                                                                                                                                                        1. 3

                                                                                                                                                          The server is going to serve contents, and applications could be half-english half- because of how language and system libraries got entangled together, which is sad.

                                                                                                                                                          Finely-designed applications should not need these. Servers not serving content directly to an user should not need these. Writing programs that serve content to the user and support various languages could need these.

                                                                                                                                                          1. 2

                                                                                                                                                            Sysops are human, too.

                                                                                                                                                          2. 4

                                                                                                                                                            It’s not simplicity when you’re creating artificial barriers for common problems that have already been solved. I understand only wanting to officially support English, but cutting out all language support otherwise is not the same thing. It’s an almost hostile decision.

                                                                                                                                                            1. 2

                                                                                                                                                              KISS is supposed to be a simple and minimal base distribution, with the philosophy that it’s easier to add software than to remove it. The author has no need for internationalization, so therefore it’s not included, but the package manager is designed so you can add other repositories and have them function no differently from the “official” one.

                                                                                                                                                            1. 1

                                                                                                                                                              I bet E-mail has the same problems. I’m skeptical that public personal communication benefits particularly from an open, decentralized format.

                                                                                                                                                              1. 1

                                                                                                                                                                I think it mostly comes down to using domain names as part of the user ID.

                                                                                                                                                                If you want to pick a provider, you will probably choose the one your friends use. This is first of all caused by the fact that, since it’s part of the user ID, you automatically know which provider your friends use, and your friends will know which provider you use, and you want to fit in. Also, when picking a provider, since the provider name is a part of your name, you need to make sure you pick a provider with good longevity, and new companies have terrible longevity compared to old ones. Thus, the popular nodes get more popular, and the obscure nodes remain perpetually obscure.

                                                                                                                                                                1. 2

                                                                                                                                                                  So, I opened issues asking about multi-domain support on all of the major ActivityPub servers about a year ago - so far, interest has ranged from ‘absolutely not’ to a single instance of ‘would consider merging a PR’.

                                                                                                                                                                  IMO cheaply separating the host from the domain is one of the most important steps to support federation. For instance, I have my own domain for email, but if I had to run my own server to do so I’d probably pack it up and go back to google. MastoHost offers hosting from €7 per month, which is a little high given how little I use social media.

                                                                                                                                                                  They can’t push prices much lower, though, because (assuming they aren’t violating the AGPL by keeping their own patches in-house) they need to run a separate process for each domain in use, which eats up memory pretty quickly.

                                                                                                                                                              1. 11

                                                                                                                                                                I realize that I’m responding to a different post from this same author, but it seems important enough to call out:

                                                                                                                                                                Now let’s be clear here - this addon [LibreJS] has nothing whatsoever to do with privacy, functionality, convenience, or anything benefiting the user. It only serves to satisfy a particular brand of autism called freetardism.

                                                                                                                                                                *sigh* So much problematic packed into one sentence, I’m curious whether the concept of “freetardism”, or the use of “autism” as an insult, would get more lobsters up in arms. This post also makes a point of referring to companies as “Evil”. It’s obvious enough that the writer is intentionally using charged language, so I would like to downvote the OP as troll for it.

                                                                                                                                                                I additionally take issue with this mindset:

                                                                                                                                                                By using them, you are also relying on someone else to provide you with the lists, instead of taking your web browsing into your own hands.

                                                                                                                                                                Besides the obvious fact that this is false (you can write custom blocklists in most ad blockers, including uBlock Origin), it’s also ineffective for a fairly obvious reason: custom blocklists are a terrific way to fingerprint someone’s browser. You are making yourself easier to track when you use non-standard blocklists, and you’re putting in a lot more work to do it. That’s why Tor Browser doesn’t include an ad blocker.

                                                                                                                                                                Probably the best way to protect your privacy, assuming that Apple isn’t blatantly lying about what their software does, is to use Mobile Safari on a fairly-recent iPhone, constantly be clearing your cookies, don’t install any third-party apps, and don’t take the iPhone out of your house (don’t want it to track your location, after all, while the location where you live is already known to Apple and your LTE provider for billing and warranty purposes). Since iPhones are super-popular, basically identical to each other, and don’t make any active effort to tie themselves in with Google’s tracking system, they’re really hard to fingerprint.

                                                                                                                                                                I’d put Tor Browser in a distant second. If it’s a niche website, you might be the only visitor who uses it, which itself is a fingerprintable fact, though obviously it’s hard to figure it if you’re actually an individual or not.

                                                                                                                                                                1. 3

                                                                                                                                                                  I looked through his page before posting the link, and while I didn’t like articles such as “Refuting Freetardism”, I think the the page as a collection of links and references had a greater value than the particular opinions the author holds on these issues. Other than that, the language he uses seems like pretty standard /g/-speak, for whatever that’s worth.

                                                                                                                                                                  1. 9

                                                                                                                                                                    Yeah, but on the other hand, this guy’s recommendations are probably a bad idea. Online anonymity is a statistical problem; what you want is to make it as hard as possible to sort online browsers into buckets. If I can recognize someone as that one guy browsing Lobsters using Pale Moon on a computer with a hidpi display, then you’ve failed. By this logic, clearing your cookies is a good idea because we need as many people to have identical sets of cookies as possible, and none makes a convenient Schelling point. Similarly, blocking tracking scripts is mostly a performance optimization, not a privacy enhancement, though you can minimize the induced harm by trying to make sure people block the same set of tracking scripts, either by having some fixed rule about third-party scripts or a shared “artificial Schelling point” like EasyList. Also, webmasters and ad companies are not perfectly-colluding Rational Actors with no motives other than tracking you, so blocking these scripts can help with your anonymity by depriving the adtech companies of a way to track you sans cooperation from the content provider. But since websites are already known to do work to detect ad blockers, and will in some cases allow ad providers to proxy their site, custom blocklists are at best a mixed bag, and if you want to pick from a mixed bag, you might as well pick the one that doesn’t require tons of work on your part.

                                                                                                                                                                    p.s. This also means that privacy is not opposed to ease-of-use. Ease of use is required for privacy, in order to ensure that your configuration doesn’t reveal your technical expertise.

                                                                                                                                                                  2. -2

                                                                                                                                                                    sigh So much problematic packed into one sentence, I’m curious whether the concept of “freetardism”, or the use of “autism” as an insult, would get more lobsters up in arms. This post also makes a point of referring to companies as “Evil”. It’s obvious enough that the writer is intentionally using charged language, so I would like to downvote the OP as troll for it.

                                                                                                                                                                    The definition of what language counts as charged is polticial, and characterizing the entire post as trolling for using word choices you don’t approve of is an attempt to use the formal structure of the troll downvote mechanism to suppress the entire post for not adhering to your speech code, which I disapprove of.

                                                                                                                                                                    1. 1

                                                                                                                                                                      Insults are not covered by free speech.

                                                                                                                                                                  1. 7

                                                                                                                                                                    Stop recommending GPG.

                                                                                                                                                                    1. 7

                                                                                                                                                                      Stop recommending to stop recommending GPG. GPG is difficult and absolutely has it’s sharp edges, but it is also “standard”. I use it every single day in both a corporate environment, personal use, and a ton of places in between. The article you link fundamentally misses one of the main reasons almost everyone uses GPG, encrypted email. I do a ton of vulnerability disclosures and mailing the security@COMPANY.WEBSITE with a GPG key and a vulnerability notification is the only consistent way to safely get my communications across. I’ve dealt with s/MIME, home brewed crap, third-party web portals, and a ton of other things. GPG is the only usable thing in the space that I have and I’ve never seen a successful migration away.

                                                                                                                                                                      1. 3

                                                                                                                                                                        Stop recommending to stop recommending to stop recommending GPG.

                                                                                                                                                                        For one thing, it’s not as simple as whether the tool is “a good thing” or not. If your goal is to use an existing email address with cryptography, there’s probably no better way to go about authenticating a message than what GPG does. If you really do need it, then obviously you should use it. If you’re able to employ it with enough success that getting error messages is actually a sign of intrusion, rather than being seen as a sign that you messed something up, then it’s doing its job.

                                                                                                                                                                        The question, of course, is whether running cryptographic secure communications over existing email infrastructure, or something very much like it, is actually a requirement that most people have. It is for you, because you’re constantly sending unsolicited messages to people you have no preexisting connection with. So the value of using “standard” communication channels is greatly heightened, compared to people who mostly communicate with friends, family, and coworkers, and probably prefer using communication channels where both sides have to open a gateway to contact each other (look ma! no spam!). If you’re using a communication channel that requires such an explicit opt-in, then that opt-in stage is the perfect place to perform key exchange while you’re at it.

                                                                                                                                                                        Also, a lot of use cases where PGP is currently employed would be better served with other tools. For example, if I was God King of Debian and had the chance to redesign their package management system, I’d probably build their package signing on top of libsodium instead. It’s actually intended to be embedded in other applications: it has a far better API, a far simpler design, and there’s really no point in using a “swiss army knife” CLI when it’s being invoked through Debian-developed wrapper tools approximately 100% of the time anyhow.

                                                                                                                                                                        1. 6

                                                                                                                                                                          There is a difference between categorically saying “stop recommending GPG” versus “check to make sure GPG is what you need and that there isn’t an alternative”. I stand by my first negation, GPG has it’s place.

                                                                                                                                                                          Whether I or you likes it a lot, the vast majority of the corporate world in the US (and outside) uses e-mail as it’s primary forms of communication and because of that I have to do things like deliver reports, exploit PoC’s, breach notifications, etc. that are absolutely sensitive. If my only form of contact with those organizations is e-mail, then what exactly are my options? Because GPG is “standard” for all of those use cases. Any mature organization I work with has had at least one security point of contact with a GPG key that can be used for further confidential conversations. I’d love to get rid of email, but let me tell you, if you try force your preferences onto another organization you are going to have a bad time.

                                                                                                                                                                          I’m in totally agreement about package management signatures being a not so great place for GPG, but that’s why I mention sharp edges. It’s not a swiss army knife, but I think much of that is the fault of apt/dpkg as it is GPG’s.

                                                                                                                                                                      2. 4

                                                                                                                                                                        What do you recommend in place of it?

                                                                                                                                                                        1. 7

                                                                                                                                                                          Check this list out, it seems pretty good https://blog.gtank.cc/modern-alternatives-to-pgp/

                                                                                                                                                                          1. 1

                                                                                                                                                                            I hope saltpack gets more attention. It seems like the perfect drop-in replacement.

                                                                                                                                                                            1. 1

                                                                                                                                                                              I don’t like that keybase seems to be the only thing developing/pushing it. Also:

                                                                                                                                                                              What state is the project in?

                                                                                                                                                                              It’s a draft and being tested in the keybase alpha app.

                                                                                                                                                                        2. 2

                                                                                                                                                                          It’s one of only a few tools NSA said they couldn’t break. They were breaking many other things people are using. Using a subset of it to just encrypt and decrypt files containing messages is easy enough for even lay people. Can be scripted, too.

                                                                                                                                                                          Given NSA > most other threat, using GPG will probably handle them, too. So, I prefer it for proven effectiveness. The attackers will probably get me via Firefox before it.

                                                                                                                                                                          1. 1

                                                                                                                                                                            Using a subset of it to just encrypt and decrypt files containing messages is easy enough for even lay people

                                                                                                                                                                            This is misleading, gpg interface is notorious for ease of misuse.

                                                                                                                                                                            Given NSA > most other threat

                                                                                                                                                                            If this is your threat model, then it’s more about opsec than specific tools. Check out grugq guide on operational GPG for email, for example, it’s quite tricky to get it right every time.

                                                                                                                                                                            1. 1

                                                                                                                                                                              “gpg interface is notorious for ease of misuse.”

                                                                                                                                                                              People been repeating that for years instead of mitigating it. I wonder why given how easy it is. You create a cheat sheet with just a few items on that one, add the good options for key gen phase, and do something about the painful encrypt command. A shell script or something so they can type less stuff in. Then, you’re good.

                                                                                                                                                                              “If this is your threat model”

                                                                                                                                                                              My threat model is people breaking crypto. I also prefer vetted solutions. The NSA vetted this one in Snowden leaks. Most others they broke. If it causes them problems, it should work well against the lesser attackers most people are concerned with.

                                                                                                                                                                        1. 2

                                                                                                                                                                          Is t here any way to protect your site if it’s being served through a CDN? Not just the site’s resources but the pages and everything.

                                                                                                                                                                          1. 5

                                                                                                                                                                            A proxying CDN like Cloudflare? No, they’re a voluntary man-in-the-middle for your website.

                                                                                                                                                                            1. 2

                                                                                                                                                                              If you’re using something like CloudFlare for your pages, then as far as the browser is concerned CloudFlare is your site.