1. 27

    What a delightful example of everything wrong with modern publication practices:

    In some cases, I won’t really read the rest of the paper if I’ve already decided it’s getting The Big SR.

    Yeah, God forbid something with a rough intro but an interesting proof make it into the common body of literature!

    Focusing the paper on the mundane implementation details, rather than the ideas.

    It’s precisely the mundane details that tend to be important in the real world. Especially in computer science, where people are unmotivated to provide useful experimental data or source code to duplicate they’re work, these mundane details are the only chance a later reader has of reproducing or learning from their work.

    If by simplifying the problem just a little bit, you render your beautiful design unnecessary, it might be time to work on a different problem.

    Taken to its conclusion, this sort of haughty dismissal does kill off a lot of interesting work. Sure, there is a lot of research done that seems to be an answer in search of a problem, but that’s common to a lot of early engineering fields. Hell, quaternions in math were basically practically useless until aerospace and later computer animation.

    ~

    Some of the complains (bad grammar, ignoring related work, and being overly verbose) are valid, but goddamn does the author sound like one of the folks that kills papers in peer review unfairly.

    I understand that it’s thankless work, but really that’s a problem that should be solved with the system (perhaps by, I don’t know, removing the gatekeepers).

    1. 18

      Yeah, God forbid something with a rough intro but an interesting proof make it into the common body of literature!

      There are already far more papers published every year than anyone could ever hope to read even the intros. If you’re submitting something for publication, show some respect for your readers.

      It’s precisely the mundane details that tend to be important in the real world. Especially in computer science, where people are unmotivated to provide useful experimental data or source code to duplicate they’re work, these mundane details are the only chance a later reader has of reproducing or learning from their work.

      You should include all the details needed to reproduce… in the appendix. Not in the paper itself.

      Taken to its conclusion, this sort of haughty dismissal does kill off a lot of interesting work. Sure, there is a lot of research done that seems to be an answer in search of a problem, but that’s common to a lot of early engineering fields. Hell, quaternions in math were basically practically useless until aerospace and later computer animation.

      If the research is purely theoretical that’s ok - but say so. Coming up with a completely contrived “application” doesn’t help anyone.

      1. 7

        Regarding the mundane details, some of them are very much inconsequential. Was it a pentium running at 300mhz or 350mhz? Who cares? If the results are meaningful, I should be able to reproduce them on my 400mhz pentium. There’s a lot of this in 90s era papers. Absurd detail in describing the hardware, but only pseudo code for the actual code.

        1. 7

          Counter argument: was it a 486SX or 486DX? Was it a Pentium III or Pentium 4? Those are meaningful differences.

          If they do not list their exact experimental setup, they are not doing reproducible science.

          1. 7

            I like the information in principle, but in practice, at least for papers intended to have some kind of non-ephemeral lifespan, I rarely actually find it usable for reproducing. When I run into a paper that goes on for paragraphs about the finer points of their Sun UltraSPARC II cluster, what I am supposed to do with that information in 2016? Start buying parts on ebay to replicate their results? I do think they should at least briefly report what they ran the experiment on, but it’d be much more useful to structure the experiment so that it’s not so tied to details of a specific hardware platform, at least in cases where the tying isn’t inherent to the problem (certain kinds of low-level OS research can’t avoid it).

            That’s been one sort of side benefit of the move to the cloud. Lots of researchers now run their experiments on the cloud, and because you can’t really trust that the exact performance characteristics are reproducible, people try to not rely on them when possible. For example in a search algorithm or neural-net training, count some reproducible hardware-independent feature (node expansions, training iterations, etc.) rather than wall-clock time, since that’s more reproducible on different machines and the results don’t end up dependent on things like how loaded the cloud was at the time you ran the experiments.

            1. 4

              I mostly disagree: it’s the time/space complexity that usually matters. A linear-time algorithm scales linearly on a 486SX or 486DX. I think the primary exception is you want to show a new optimization that requires particular instructions. But the far majority of work that list exact machine specifications are not in that area.

              From practical experience: if you leave the exact details out, another reviewer will complain that you should specify the machine exactly ;).

              1. 3

                A linear-time algorithm scales linearly on a 486SX or 486DX

                If you’re only interested in asymptotic complexity it’s not appropriate to put any kind of benchmark numbers into your paper at all. It is a fine and lovely thing to publish papers about asymptotic complexity.

                For papers where you want to present a technique that you think makes an improvement on running time on real-world hardware, I’m going to want to know roughly what the hardware that you used looks like.

                It’ll matter for applicability. Say I read a 10-year-old paper about how someone managed to make a program go 40% faster by avoiding the need to make some syscalls, then it’s easier to judge the likelihood of it being applicable to the hardware that I have on my desk right now if I know whether the benchmark was performed on a Pentium 4 (which had the slowest syscall implementation on any CPU I’ve ever heard of) or on an Itanium (which apparently had unusually fast system calls).

                1. 2

                  I think the primary exception is you want to show a new optimization that requires particular instructions.

                  So, the problem is that in many cases you can run into implementation factors that occur even without seeking particular instructions.

                  My example of the Pentium III and P4 is exactly such a case: One could present a hashing or crypto algorithm that has better theoretical performance in their paper, and then during testing on their P4 find that it fails to be much faster than the previous generations' work on the subject (which happened to use PIIIs). The reason being, in this example, the hardware barrel shifter that was on the Pentium III but not the P4–and thus, for shift-heavy workloads like crypto, you’d expect worse performance. However, you’d only know about this if the authors of both papers showed their experimental setups.

                  There are similar issues with graphics cards, wonky hard drive controllers, and all sorts of things. On the whiteboard, sure, every algorithm is obvious, but you don’t have engineering happening until you care about implementation.

                2. 2

                  Or not? I’m not running any of those processors today. So should I just throw the paper out? If the science is useful, I should be able to reproduce it on my computer.

              2. 6

                I understand that it’s thankless work, but really that’s a problem that should be solved with the system (perhaps by, I don’t know, removing the gatekeepers).

                What gatekeepers? If you’re getting a paper peer previewed, then you’ve already submitted to going through gatekeepers by definition. But there is nothing stopping you from publishing your work without going through a peer review system, and plenty of places/people do that (unfortunately they are usually cranks).

                I’m not actually sure what world you want to live in. In anything where you submit to other’s views and opinions there is going to be a bunch of bullshit, and sure that needs to be worked on, but I’m not sure what you want to be done. Removing fictional gatekeepers doesn’t make research better.

                1. 5

                  Let me rant a little about the system,because im afraid it will take a while to fix:

                  The scientific system is truly broken nowadays.

                  I believe some massive intervention is needed. Yes, you have the entitled reviewers, like OP, but you also have the circles of friends that don’t review at all.

                  But Peer review is just the tip of the iceberg: the amount of shitty papers is overwhelming, but everyone need to publish something to justify the research founds received. Or to get more grants. And the best way to get grant money is to invent some glorious BS, and publish 4 content empty pages on a high impact factor journal. And fuck statistics and scientific method, if you want to do some research you do need to publish more, so who cares about reproducing the results. There’s not enough time anyway. And consider that theres no international organization that really matters to judge misconduct. And fines and punishments are ludicrous. And that if the authors want a paper retracted because the professor changed it and lied it takes something like 4 years. And whistleblowers will never get hired again in academia. And the science is done by postdocs and grad students that work 70h week for 35k$ year, with a good chance of being blackmailed on their visas if they don’t produce enough.

                  Sorry for the salt, but fuck science guys. until we manage to decouple it from money and fame it’s only going to get worse.

                  1. 3

                    I understand that it’s thankless work, but really that’s a problem that should be solved with the system (perhaps by, I don’t know, removing the gatekeepers).

                    When reading through this I just kept thinking, “Why isn’t this on stackexchange, it would be similar to codereview.stackexchange.com?”. Git would also be a reasonable way to handle this. The general public could submit merge requests for spelling and grammar. The real reviewers could comment on bad lines or submit merge requests to make improvements. You get a history/audit trail for free. If you use github you can even do the whole thing from start to finish online without running anything locally.

                    1. 10

                      You can do all that before submitting the paper. Your paper should be proof read before submission, not after.

                      1. 2

                        Well it could be like an issue on a kanban board where it progresses from multiple drafts to public request for comments, real peer review and finally publishing. Updates after that could even be issues.

                        1. 2

                          But that would require interest, cooperation, openness and most of all, agility. It is going to happen, but not sooner than in 20 years.

                          Right now, scientists are having a hard time establishing repositories for sharing raw data. Apparently, it’s quite hard when you are competing. Sharing, that is.

                          And, in some cases, the data are outright dangerous to publish directly. Czech linguists apparently have recordings of some elder roma talking about the time they murdered someone. Imagine publishing that when your colleagues work on their integration…

                      2. 5

                        It’s pretty common in some areas to first post a preprint on arXiv to solicit feedback. People are usually not interested in proofreading your paper, but if it’s in their area they may have questions about the work, requests for clarifications, complaints about its treatment of existing work, etc. In physics that kind of pre-peer-review has become close to standard, and papers only get submitted to journals further down the line, once they percolate through the arXiv a bit first.

                        1. 2

                          I wish it was possible.. Maybe internally to a group, but globally will not work without major reforms. Because of money, fame, pride, distribution and different audience, and much more: You need to have a precise author list if you want the $. You need to write in English if you want to reach out. Problems are usually subtle when comparing to common code samples. And they don’t necessarily have a beginning or an end.

                          Also Consider this: If science were really scientific you would not have to use literature to describe your work, but formulas and data would suffice. (Like in code). Sadly, very few fields actually let you do that, and those are usually very sparsely populated.

                          1. 4

                            formulas and data would suffice. (Like in code).

                            The comparison here would be code without comments or documentation. The formulas and data might be the important bit, but the words are also valuable so that the other people don’t have to reverse-engineer what the heck it is you’re on about.

                            1. 3

                              Sure. But technically shouldn’t a good paper follow a very strict format, like a form or a code page? Or be a simple pull request to the State of the art?

                              Let’s say, for the sake of the discussion that I’m a young researcher and i have realized a solar cell with +x efficiency vs state of art.

                              There’s not much more to it, I might have changed materials, preparation or whatever. As a scientist i am feeling obliged to share my work. ideally i should publish just an update to the state of the art. fill a form/diff or something like a pull request.

                              Instead, first i have to spend one month wrapping it with crap. Then it might be rejected because the reviewer is too busy to unwrap or doesn’t like my oratory skills .

                              I really don’t understand why we bother with the artistic quality and reject over novelty and plagiarism.

                              Imho the ideal scientific journal should be in between a git repo and a wiki. We all could be much more productive without the time wasted writing ~~poetry~~ papers.

                              1. 4

                                Writing is an important step in clarifying your own thoughts, and in subsequently communicating them to others. I emphatically disagree that time spent writing in prose of reasonable quality is a waste, or just some kind of peripheral artistic indulgence.

                                We’re not machines, but human beings! Literature is one of the tools we have built as part of the human process of science, engineering, and other professional fields: communicating with all the other humans.

                                1. 1

                                  One of my former colleagues was a very good writer, he had a lot of good publications. Of absolutely no real content whatsoever.

                                  I obviously enjoy reading great papers too, but you must admit that nowadays too often the data/content is subordinate to the prose.

                        2. -1

                          Do you think this is a serious post? I first read it as some kind of self deprecating satire by pretending to be tone-deaf in their declaration of self-superiority - I mean who would write like this in ernest, but one never knows …

                        1. 1

                          At work: Finishing up writing a fairly boring thing in Go that’ll run in a cron job and replace some of what we’re doing with Chef. It’s not a great fit for chef - we just need to do some housekeeping work on every host every few minutes. Using chef to get a simple binary onto the host and scheduled in a cron job during our provisioning process seems like a much cleaner solution in the long run than making chef deal with this every few minutes, especially considering the nature of the work means that what the job needs to do is unlikely to change often (actually, probably never. I expect almost any future changes will just be bugfixes and those should be fairly infrequent as this thing really is very very simple.)

                          At home: I was working on a proof of concept of a distributed systems framework which just got a near-total rewrite as I realized that my plugin system was totally bogus because I didn’t propagate the everything-is-versioned concept far enough into the architecture - everything was a plugin and plugin implementations were versioned, but plugin interfaces weren’t. Dumb oversight. Now really everything is versioned (except the tiny kernel bit that loads the PluginLoader plugin at startup…) and literally every component can be (mostly) safely upgraded in the running process.

                          Now I just need to port all my old plugins to the new system and I’ll almost be back to where I was a month ago…

                          I’m also playing way too much Factorio. I am pretty sure there were logistics robots in my dreams last night.

                          1. 7

                            IMO It pretty much just doesn’t matter. Even in languages without significant whitespace, you still (generally) indent your blocks. Python especially would look basically identical if it used curly-braces instead of whitespace.

                            The only problem I have with significant whitespace languages isn’t one with the whitespace or the language, it’s with my stupid brain: I always forget which languages use which style when switching between them. I usually write a lot of Python, but the current project is in Go so about 100 times a day I start a block with a colon and an indent and then have to go back and fix it when gofmt yells at me. Then again, I do it with other stylistic things too, like accidentally naming a function do_thing instead of DoThing.

                            1. 9

                              Doctors don’t need to beat down other doctors and climb management hierarchies just to advance; they just need to gain more skills and deliver more value.

                              This is not what I’ve observed with professionals in medical communities. In fact, the only thing keeping the docs from going completely ape is that they have the permanent underclass of nurses to beat on.

                              That is, a doctor can’t use the superior orders defense to justify killing a patient

                              Demonstrably false, regrettably. Nurses and others do this, though they are typically tasked with doing exactly what the physician wants or is reported to want, even if it means giving a patient a fatal dosing of an antibiotic.

                              Doctors may also do the same thing in different guises. It’ll be dressed up in terms of “standard of care” and whatnot, but that’s just clever marketing.

                              ~

                              I believe that there should be some kind of professionality in software engineering (if it is indeed ever to become a form of engineering), but basing ourselves on the practices of law and medicine is maybe not a good move.

                              1. 5

                                This is not what I’ve observed with professionals in medical communities.

                                There’s politics, but it’s not at the same level of intensity, and the consequences aren’t as severe. In the true professions, losing politically means that you get a worse office and, if things go really bad, you might be expected to find another firm in the next 5 years (the “succeed elsewhere” talk). For programmers, it can mean being suddenly fired for no good reason or just no reason.

                                Demonstrably false, regrettably. Nurses and others do this, though they are typically tasked with doing exactly what the physician wants or is reported to want, even if it means giving a patient a fatal dosing of an antibiotic.

                                Are you talking about euthanasia? That, while illegal in this country, isn’t what I was talking about. (Why would anyone use antibiotics for euthanasia, though? Seems like a painful way to go, and risky as hell for the doctor.)

                                basing ourselves on the practices of law and medicine is maybe not a good move.

                                That may be true. Medicine requires expensive schooling and isn’t really an option after a certain age (I don’t think that a 35-year-old would get admitted to med school) and law, on the other hand, opened its doors too widely and now has a large underclass of people with third-tier law degrees. I think the actuarial sciences have the best model: difficult exams, no required schooling, and the ability to dynamically control the growth of the profession (through exam difficulty and pricing, in part) according to economic circumstances.

                                1. 5

                                  Are you talking about euthanasia?

                                  I presume they’re referring to nurses failing to catch (or noticing but failing to prevent) doctors' mistakes, such as prescribing a fatally high overdose of some medication.

                                  However, this is cherry-picking anecdotes. There are stories of nurses ignoring or modifying unusual (but non-mistaken) instructions from doctors to the patient’s detriment as well. In general, doctors make medical decisions because they are trained to do so. Nurses do not because they are not.

                                  1. 2

                                    Okay. Yes, that makes sense having reread what he said.

                                    Still, don’t the nurses at least ask the doctor when the numbers look off? I mean, I wouldn’t necessarily hold a nurse legally responsible, but I can’t imagine that a nurse would get in trouble for asking a doctor, “did you really mean 200 mg and not 20 mg?”

                                    1. 1

                                      They certainly shouldn’t get in trouble, and it’s in the best interests for almost any org even those outside medicine to encourage people to call it out when they see something weird and especially something dangerous. I think in IT we have a lot to learn on the subject - “Crew resource management” procedures from aviation are a particularly good example of doing it mostly-right.

                                      Not particularly relevant, but sorta related, there was this great article examining an occasion when technology and operational procedures went haywire to create a dangerous situation for a patient: The Overdose: Harm in a Wired Hospital. It’s long, but really interesting.

                                      1. 1

                                        Yeah - I see this is an earlier article in the same series as the one I first thought it was. I particularly liked this later one, comparing how it’s handled in healthcare and aviation. Should Hospitals be more like Airplanes

                                        I definitely think finding ways that we can get actual reliability - and acknowledging and dealing realistically with alert fatigue is a huge part of that - is a big part of what we need to change as programmers, to stop relying on this whole heavy-management thing for fake reliability.

                                        1. 5

                                          I definitely think finding ways that we can get actual reliability - and acknowledging and dealing realistically with alert fatigue is a huge part of that - is a big part of what we need to change as programmers, to stop relying on this whole heavy-management thing for fake reliability.

                                          Amen. System reliability comes from individual excellence. We obviously need both excellence and reliability. Humans are just really bad at the latter. Often, it requires human excellence to build systems with high technical reliability.

                                          I think that the micromanagement culture comes from this idea that superficial industrial reliability is a good proxy measure for ethical reliability (which matters, and on which one can’t compromise). It’s not.

                                  2. 1

                                    Are you talking about euthanasia?

                                    Story I was thinking of was a neurosurgeon talking about one of their learning surgeries. If I recall correctly, the teaching physician instructed the doc to do something rather risky for the hell of it (theoretically, to train them on something). Had it gone wrong, patient would’ve been partially paralyzed or dead, and the defense used would’ve been “I was told to do this.”

                                    Malpractice is actually a whole set of misadventures: https://www.texasobserver.org/anatomy-tragedy/ .

                                1. 0

                                  Can anyone else not wait to see RMS’s response? I imagine it will be… not favorable. And that he’ll demand we call this GNU/Windows rather than “Windows Subsystem for Linux.” Or maybe “Windows subsystem for GNU?” I can guarantee he’ll be salty it’s called anything to do with linux when it isn’t even using the linux kernel.

                                  1. 7

                                    I find it very odd he’s completely chucking out PowerShell. PowerShell, both as a scripting language and as an interactive shell, is actually one of the best environments I’ve ever used. It’s definitely not perfect, but it honestly gets a lot of things right. The trivial extensibility from .NET, the entire remoting/workflow system, things like Out-GridView, the Interactive Scripting Environment (ISE) for writing scripts…seriously, they really got a hell of a lot of things right.

                                    I’m really excited to have bash on Windows because it means that bash is now the lowest common denominator for a quick script (v. writing separate PowerShell/batch and bash scripts), but if you’re just talking about day-to-day usability, I don’t actually think bash helps a ton.

                                    1. 9

                                      Author here.

                                      I’m not sure what you mean when you say I’m completely chucking out PowerShell. I called it “the cure for polio” and Jeff Snover “the Jonas Salk of the Windows ecosystem”. To be completely honest, I feel like I was a little hard on Bash if anything.

                                      1. 5

                                        I think I took the first at least really differently; in context, I took it to mean “a great cure for something else” (i.e., fixing the wrong problem). I’ve heard a lot of devs say that (“it’s a better WSH, but we didn’t need a better WSH”, for example, or “It’s a better shell, but the console subsystem is still crap”, and so on), so maybe that’s where my head was at. The post being called “the Windows command line,” and not “cmd.exe”, seemed to cement that.

                                        At any rate, I wasn’t trying to mischaracterize your writing. People always glom onto random parts of my posts, extracting a meaning I not only didn’t intend but actively disagree with. Sorry I was the one doing it here.

                                      2. 1

                                        I don’t think he’s throwing it out per se, I just think he’s not really talking about it (and using a slightly click-bait-y headline.) I mean at the end he says your choices are batch, bash (now) and powershell. Fairly clearly only one of those is not the right choice anymore.

                                      1. 3
                                         % Notepad.exe
                                         zsh: command not found: Notepad.exe
                                        

                                        (-̩̩̩-̩̩̩-̩̩̩-̩̩̩-̩̩̩___-̩̩̩-̩̩̩-̩̩̩-̩̩̩-̩̩̩)

                                        1. 2

                                          Exactly.

                                        1. 2

                                          Starting a new job today so… No idea!

                                          In my personal time I’ve been playing around with building a python distributed systems framework to make it easy(ish) to build programs that scale. I don’t have a huge amount to say about it yet, but the core idea I’m working on is building versioning deep into the system and message protocol with the main intent being to make large-scale deployments easier. I’ve found one of the biggest challenges in really big systems to be deployments - particularly because it can take days or weeks to complete one, so the system uses versioned plugins for pretty much everything. New plugin-versions can be loaded into a running node, so hotfixes are zero-downtime. “Real” deployments are also made easier as the old version of a plugin continues to be used until the new version is requested, so you don’t run into the problems of having different nodes running different-and-maybe-incompatible software versions.

                                          One of the most fun things about it right now is that even the message protocol handlers are plugins, which use the nifty trick of ‘respond in the protocol-version that the request came in, unless the request explicitly asks for another one.’ So most of the time messages use a binary format for efficiency but multiple protocols can be supported - this is super handy for debugging as you can just send it messages in a human-readable protocol and get a human-readable response back.

                                          The plugin-nature also enables some other neat debugging tricks, like you can load a debug version of some plugin with a “breakpoint” that starts a remote-pdb session so you can remote in and inspect the state of the node.

                                          Right now I’m working on making the programming model a little easier for the thing, as right now it’s a bit tricky. Pretty much every operation you want to do requires asking the plugin manager to give you the thing that handles that operation with some version spec, building up a wonky data structure as an input to the plugin, and then invoking the operation on that plugin. It’ll be a lot nicer if I add some Spring-ish dependency injection magic to make getting ahold of the plugins easier, and maybe some magic to make invoking the plugin look more like, you know, a method call.

                                          1. 2

                                            Orgmode baby. You can export to many formats.

                                            1. 2

                                              org-mode is just totally glorious, unless you happen to work with people who don’t/won’t use emacs. It’s my tool of choice for authoring stuff when I don’t need to collaborate, though.

                                              That said, I did once work with a crazy guy whose love for org-mode was only equaled by his love for vim, so he just wrote org docs in vim… Most of his editing would be done that way and then he’d boot up emacs whenever he needed to do exports or recalculating tables or something. It was weird.

                                              1. 1

                                                I draft in org mode anyway, then export to whatever format I need for collaboration once I’m near done. Gists render org very nicely :-)

                                                Sounds like your crazy colleague needed to discover evil mode :-)

                                                1. 2

                                                  Actually I showed him evil, but he didn’t like how emacs took longer to start than vim. Then I showed him how to run emacs as a daemon and connect with emacsclient. Then, finally, he admitted it was just an irrational preference thing, and I went back to my office to let him vim away in peace.

                                                  1. 1

                                                    Some people can’t be helped.

                                            1. 27

                                              The absolute most important criteria (for me, anyway) is missing from this analysis. Which format leads to more documentation being written? For me personally, I loathe reST and I get its syntax wrong all the time. Comparatively, Markdown is intuitive, simple and easy for me to write. For me, it removes a barrier to writing documentation, which increases the probability of me writing documentation. It wins on that criteria alone.

                                              I know it’s not just me who thinks this way. rustdoc is purely Markdown. godoc doesn’t even support Markdown, and instead mostly renders only plain text with a few very small niceties. In other words, it gets out of your way.

                                              1. 5

                                                So this is just some insane rambling, I couldn’t sleep last night so please forgive me.

                                                In my experience “how much documentation is written” has a lot more to do with culture than format. This is why, for example, I really overall dislike wikis.

                                                One story I sometimes tell is that the only company I’ve ever really been happy about the documentation was when I wrote basically all of it. The situation was I was more-or-less the last technical person at the company and needed to help re-bootstrap a sysadm and support team and so needed materials for them for training and reference.

                                                The format didn’t matter much, since I was writing most of it I used emacs org-mode and exported to PDF. The PDFs were distributed out of sharepoint.

                                                What turned out to be a lot more important was the culture I established - that documents had owners and their email addresses were listed at the top of the doc. If anyone had questions or found an error, sending those issues to the doc owner was the right thing to do, and it was a significant part of my job to do the upkeep on the docs I owned. This worked really well in the short term, as really it was for my own benefit. As the last tech guy, I was oncall basically 24x7x365, which was totally untenable. The first docs I was writing were SOPs for common issues so we could offload some of the oncall responsibilities to the other teams, and just have them escalate to me when the SOPs didn’t cover the issues so I was pretty motivated to get all the feedback I could to make sure that those docs were as high-quality as possible, as when they had problems I didn’t get to sleep through the night.

                                                What surprised me was that in the longer term when I thought everyone else would start using wikis and such, that strategy was still really successful. At that point it was just the normal thing at that company and we’d trained everyone into it, so when they started writing documentation they’d follow my system (though generally authoring using something other than emacs…) The docs stayed really high quality, and surprisingly well up-to-date because we made sure that updating docs was understood to be a job responsibility along with everything else. Devs even included documentation time in their sprint planning estimates (which, you know, should be SOP but I’ve seen it disappointingly infrequently.)

                                                After meditating on this experience for a while I think the big problem with a lot of docs, regardless of format or distribution mechanism (word docs in sharepoint, wikis, three ring binders, etc) the hardest thing to avoid in documentation is diffusion of responsibility / the bystander principle (and wikis are especially vulnerable to this, since diffusion of responsibility is their raison d'être.)

                                                The big difference is the approach. Often what we want to see in wikis is: “this doc looks wrong, I’ll go figure out what the right thing is and update the wiki,” but the problem is that that operation is expensive. You need to research the issue before you update the doc. And that effort may be totally wasted - someone else might already know the answer. Since it’s expensive, often a lot less progress gets made. On the other hand, “this doc looks wrong, I’ll drop a note to jrdn since his name is on the top” is really, really cheap, and the work gets routed to the people most likely to already know the answer.

                                                The other big thing was keeping the docs in source control right next to the code, instead of separately (which is why I do quite like sphinx/markdown/plain text files again over wikis.) to continue to minimize the cost of documentation, but that’s a whole separate sleep-deprived insane rant.

                                                1. 2

                                                  I don’t actually disagree with anything you said. :-) I can totally believe that culture plays a role in how likely good quality docs are going to be written. In fact, I believe there are many factors. But for me personally, annoyance threshold with the process of actually writing the docs is definitely on that list. Especially for projects I do in my spare time. (Which require significant documentation, just to name a few. For nfldb, just look at all the documentation on instance variables! It’s great! Sphinx supposedly has a way to do that too, but you either need to stick them in the doc string on the class or do some other hacks to get it working. Granted, this isn’t necessarily Sphinx’s fault, since the Python folks don’t seem to care much about documenting one’s data representation.)

                                                  At work, we do actually keep a wiki, but we use Gollum which means that your wiki is just a bunch of Markdown files in git. It works great.

                                                  At work, we do also use Sphinx for documenting our Python code. I loathe writing out the doc strings because I’m constantly looking up syntax for reST. I grant that at least part of it is because doc strings tend to contain explicit documentation of parameter lists (with types, of which the support is spotty), which can at least be partially avoided when writing docs in a language with a type system. (Many exceptions apply…)

                                                  1. 1

                                                    annoyance threshold with the process of actually writing the docs is definitely on that list

                                                    Oh yeah, formats definitely factor into that cost-of-documentation, personally I’ve just found it to be a relatively minor factor compared to the other ones at most of the organizations I’ve worked in. Getting a good format that works well definitely helps, and there is a big benefit to simplicity (read as: I can’t remember how to write reST either. Thank goodness Emacs generates most of the basic sphinx stuff for me and I only need to look it up when I need something a little weird.)

                                                    I understand the author’s frustration with markdown, as I’ve definitely encountered some lack-of-standardization problems with it, but unfortunately that’s probably just the nature of the beast. Part of the problem is the inexact standard, but another part is that there are a whole bunch of implementations. There are a bunch of reST implementations as well, and not all of them support all the same things, so you have the same sort of issues anyway. I think the main reason I’ve had the problem less with reST is more just that there are fewer documents I have reason to use with multiple implementations, more than the standard keeping everything compatible.

                                                    Yet another reason to like org-mode, which (basically) only has one implementation. No compatibility… no compatibility problems!

                                                    Edit:

                                                    I don’t actually disagree with anything you said. :-)

                                                    And just wanted to say I definitely didn’t intend for my reply to come of as a rebuttal to your comment, so apologies if it did. It was more just that your comment inspired me to indulge myself in a rant, all my coworkers have heard that one a couple times now so I thought I’d just drag it over here to introduce it to a whole new audience :P

                                                    1. 1

                                                      Yeah, I guess when I wrote my comment I was thinking a lot more about what I’m doing in my free time versus work. The annoyance tolerance is definitely quite a bit lower for stuff done in my free time.

                                                      I think the only thing I’ve been bitten on with Markdown is the fact that reddit doesn’t support the triple backtick syntax for code blocks. Other than that, it’s been pretty much smooth sailing.

                                                      Yet another reason to like org-mode, which (basically) only has one implementation. No compatibility… no compatibility problems!

                                                      I took up arms with Vim long ago, but it does appear to have a orgmode plugin.

                                                      1. 1

                                                        Yeah, there are even some efforts to bring it to phones in Orgzly and MobileOrg.

                                                        I have hopes for some of the new implementations maybe bringing org a bit more into the mainstream, but it’s going to be an uphill battle for them. The basic features in org are pretty doable, but then there’s all sorts of other cruft that’ll be a bit more annoying to maintain like the calc based spreadsheet and the literate programming stuff.

                                                        Then again, it’s probably safe to ignore most of that in most contexts, maybe it would be a good idea to try and define some standardized subset of org to target for compatibility across implementations.

                                                2. 4

                                                  Which format leads to more documentation being written?

                                                  Absolutely agreed. I understand that Markdown is a terrible language. So is English–it’s got a textbook defective orthography with pathological phoneme/grapheme correspondence. But that doesn’t mean I’m going to start documenting my projects in Esperanto or Quenya.

                                                  1. 3

                                                    This is an uncharitable interpretation of my comment. I personally am more likely to write more docs using markdown than reST, and i attribute that to finding markdown easier to write.

                                                  2. 2

                                                    For me personally, I loathe reST and I get its syntax wrong all the time.

                                                    This is probably more related to previous experience with other markups than a specific sphinx property.

                                                    1. 4

                                                      … or it’s related to the fact that some markup languages are more complex than others?

                                                      I’ve tried using Sphinx, which is an altogether separate issue that the OP. (In fact, I do, at work.) It’s so much more complex than what I think is necessary that I wrote a replacement to epydoc for my auto documentation needs. Example.

                                                      1. 1

                                                        Not my experience. I started trying to learn reST when my only previous markup experience was MediaWiki (which is unlike any of these others). I gave it a good go, couldn’t make it work for me, gave up and started using Markdown and it was so much nicer to use. UX matters.

                                                    1. 4

                                                      I really liked the Idea of whuffie in Down & Out, and I think Doctorow is doing some of his own ideas a disservice here., though it’s fairly obvious that there’s a bunch of political points wrapped up and the whuffie idea is only really being used as an analogy for them.

                                                      Meritocracy is a tautology, of course. There’s no objective measure of ‘‘merit’’ so there’s no way to know whether your society is meritocratic or not.

                                                      There’s a lot of reasons to object to aspects of meritocracy. Luck is one, how you define or calculate ‘merit’ is another, but I think this is one of those “don’t let the good be the enemy of the perfect” situations. If there’s a compelling argument for something other than meritocracy, I’d be happy to hear it, but the rejection of meritocracy based on it’s problems without a suggestion for an alternative seems incomplete bordering on disingenuous. In the general sort of usage, I’d consider meritocracy to be a good thing - it’s the idea of judgement based on demonstrated success and ability, right? If I had a team of employees and needed to promote one of them to lead that team, what criteria should I use to pick the person to promote? If it’s something other than their “demonstrated success and ability” I think we’d need to hear a really strong argument to support that idea, as the meritocratic approach seems like the most obviously correct.

                                                      But reputation is useless as a hedge against the real nightmare of a setup like Ebay: the long con. It doesn’t cost much, nor does it take much work, to build up sleeper identities on Ebay, fake storefronts that sell un­remarkable goods at reasonable prices, earning A+++ GREAT SELLER tickmarks, even for years, until one day, that account lists a bunch of high-value items on the service, pockets the buyers’ funds, and walks off.

                                                      I think Doctorow is ignoring the huge benefit of reputation systems here. Is it true that someone with high reputation can still betray you? Absolutely, But that’s the case in pretty much all inter-personal relationships. I doubt that many of us can say we’ve never personally been betrayed by a trusted friend in some way. The big win of a reputation system is one of increased information - the system can tell me that even though this person is a stranger to me and could still betray me, here’s a whole bunch of other people who said that they took the same risk and were not betrayed. It’s the difference between meeting someone in a bar and knowing nothing about them and having to find out the hard way, vs being introduced by a friend who knows them well and vouches that they’re a good person. Sure, they could still burn you, but you can enter into the relationship with more information, and IMO that’s almost always a good thing.

                                                      Let’s also be honest about the situation Doctorow describes - the long con could happen to anyone regardless of any reputation systems, including Whuffie or the ebay seller ratings or knowing the person for years. It’s not fair to say the reputation system is ineffective because it has this failure mode, because the right way to defend against that failure is another system not based on trust. Using his ebay long-con example, when I want to buy something inexpensive, the high reputation is probably enough for me to go through with it, knowing I could still get burned. If it’s so much money that it’s loss would damage me, though, we have other systems at our disposal to remove trust from the equation like an escrow service. (Of course, to use an escrow service me and the other party both need to trust that we won’t get burned by the service, which brings us right back to the reputation economy, and you can bet I’d want to know that there are tons of other people out there that have been happy with their service rather than having to find out myself the hard way.)

                                                      If you wanted to highlight the dystopian nature of Whuffie, you need go no further than this vision for Peeple. If it ever took off, it’d be a lever that the likes of Gamergate could use to destroy your’s employment and personal life, possibly permanently, just by mass-one-starring you.

                                                      (I’m going to let the random Gamergate reference slide for the most part, but will point out that that it would be equally a problem for Milo Yiannopoulos or any other controversial figure.)

                                                      Peeple was bad in a way that Whuffie wasn’t, and Doctorow himself described at the beginning of this essay:

                                                      Whuffie reflects how respected you are by the people I respect. Someone else would get a different Whuffie score when contemplating you and your worthiness.

                                                      Whuffie has a whole other dystopian failure mode, though, and that’s how it would create (well, reflect) respect-silos, like different currencies with no exchange rate. It find it unlikely that Doctorow’s network would have enough respect for the Gamergate network for their ratings to have much impact on each other.

                                                      I think this misses the real problems with Whuffie, which is that it is far too one-dimensional. It’s not respect or reputation that’s the problem, that’s just reflecting the reality of operating in society. The real problem is trying to reduce a person to a single number based on that standing in society, when reality is a lot more complex than that. A good reputation system would probably need to reflect that complexity - for example while I might not agree with some of Doctorow’s political or social opinions, it doesn’t mean I don’t respect his opinion on other topics and it most definitely doesn’t stop me from respecting him as an author. Some reputation systems like ebay’s seller rating don’t run into this problem because they’re only based around a single variable - whether or not it was good doing business with that person. Whuffie or Peeple or such other systems do need that nuance, though.

                                                      1. 6

                                                        I want to mention again that, what is strange about all this, is that it is assumed that you always must have a way to handle the fact that mutual exclusion is violated. Actually if you have such a system to avoid problems during race conditions, you probably don’t need a distributed lock at all, or at least you don’t need a lock with strong guarantees, but just a weak lock to avoid, most of the times, concurrent accesses for performances reasons.

                                                        I’m not sure I understand this paragraph. He is saying that there is no way to handle race conditions, and if you have a way to handle them you don’t need a distributed lock, but isn’t that what a distributed lock is attempting to solve?

                                                        Despite starting off by talking about the exactness of mathematics, the end result seems to boil down to this statement:

                                                        Can a process count relative time with a fixed percentage of maximum error? I think this is a sounding YES, and is simpler to reply yes to this than to: “can a process write a log without corrupting it”?

                                                        And the reasoning behind this comes down to “Just don’t make mistakes”. What about hardware issues? What if you do make a mistake? That is why they are mistakes, after all. I think this reasoning is pretty weak and there is a lot of evidence to show that you simply cannot depend on it.

                                                        1. 1

                                                          I’m not sure I understand this paragraph. He is saying that there is no way to handle race conditions, and if you have a way to handle them you don’t need a distributed lock, but isn’t that what a distributed lock is attempting to solve?

                                                          I think you mostly get it - I think what he’s trying to say is that there are other ways to handle race conditions than distributed locking, but if you’re using those then you don’t need a distributed lock in the first place or at least don’t need it to be strongly correct, as your other mechanism provides correctness.

                                                          And the reasoning behind this comes down to “Just don’t make mistakes”. What about hardware issues? What if you do make a mistake? That is why they are mistakes, after all.

                                                          When designing distributed systems, the system’s ability to function function does often depend on “don’t make mistakes” so that much is certainly true. The other part of it though is “detect and deal with mistakes” and that’s something that is possible in many distributed systems. For example, lets say you have a hardware fault which causes the clock to run incredibly slowly - this can be detected if NTP attempts to step the clock (out of sync by more than 128ms, so give up on slew and just reset the damn thing.)

                                                          It is certainly true that undetected byzantine failures are a danger in distributed systems, but that’s pretty much always going to be the case. See, for example, the “FLP” paper: http://cs-www.cs.yale.edu/homes/arvind/cs425/doc/fischer.pdf The important thing about such results as FLP is that it is impossible to eliminate nontermination but that doesn’t mean it’s impossible to make systems that can make correct progress most of the time. Paxos or other consensus protocols may fail to accept some proposal due to a variety of reasons - but they won’t accept an invalid proposal and so in the general case proposals can be retried until the system recovers, although sometimes this recovery will require detection and removal of misbehaving nodes by some other external means. Sort of like the “CAP theorem”, in distributed systems you have properties around “liveness” (ability to make progress) and “safety” (ability to not accept incorrect messages). FLP proves that in an asynchronous system we can’t guarantee liveness, but we can still be safe and live when we’re not “making mistakes”.

                                                          1. 1

                                                            When designing distributed systems, the system’s ability to function function does often depend on “don’t make mistakes” so that much is certainly true

                                                            Of course, nothing works if you do terrible mistakes, but the goal, IME, of any distributed system is to make the system less fragile. The issue, it seems, with Redlock is it makes the whole system more fragile.

                                                        1. 8

                                                          Redlock assumes a semi synchronous system model where different processes can count time at more or less the same “speed”.

                                                          This is flawed reasoning. See https://aphyr.com/posts/299-the-trouble-with-timestamps.

                                                          1. 1

                                                            You’re correct that it is a huge problem in distributed systems, but the reasoning itself is not flawed - just something that needs to be known and compensated for.

                                                            Most of these sorts of systems that depend on distributed nodes having roughly the same ability to keep track of time handle the problem by using timeframes much longer than would be affected by small deviances in clock rates or synchronization. Using NTP configured properly, for example, it’s generally pretty safe to assume your clocks are in sync to about a couple milliseconds on a LAN, so if we just build into the protocol a safety factor much greater than any reasonable deviance you can accomplish good (but not perfect) safety. For example, acquire a lock for 30 seconds from timestamp A to B, after A wait 5 seconds, do your work to prepare to commit, but abort if B is in less than 5 seconds. Unless another system in the network is so far out of sync that it believes that that lock has already expired, the system should be safe.

                                                            It does mean that your system is vulnerable to clock/network/config issues, though, so those need to be monitored. I once saw a system mistakenly configured to use some random external NTP source (which should’ve been blocked by the network config anyway, but was not) out of sync with the rest of the network by 10s of seconds. One of the nice things about NTP is that it’ll give you an estimate of how far out of sync it is - such nodes should remove themselves from service if they’re approaching safety bounds or their skew is unknown due to failures to sync.

                                                            1. 6

                                                              While your response might be true, Martin’s analysis covered it failry well by saying if mutual exclusion is important for the correctness of your system, Redlock is not a good choice. So saying things “should” be safe and hoping that your CPU doesn’t just hit a bug one day and start telling the time wrong is an insufficient guarantee. It still means that Redlock is broken (or at least that’s the claim) in the face of a known possible error mode, an error mode with known solutions.

                                                          1. 7

                                                            I’ll upvote this, because it’s worth reading–even if you really disagree with it, as I do.

                                                            In other disciplines, engineers wear an iron ring to remind them of their commitment to their profession.

                                                            I have, time and time again, railed against this comparison of software development with more traditional engineering. If you aren’t writing FEA software, pacemaker software, machine firmware, or whatever, you are almost certainly not behaving like a traditional engineer. If you develop on mobile or the web, you almost certainly aren’t doing anything with that gravitas.

                                                            We don’t have any of the technological maturity or sanity (some would say stagnation) of other fields of engineering. JS is a pretty good example of this. The lack of a meaningful accreditation and journeyman process is a good example of this. The lack of respect of our opinions by the business folks we work with is an example of this.

                                                            Those things don’t make us bad people, don’t make us less smart…it just makes us !engineers.

                                                            When I’m building a bridge, or an elevator, or a refinery, or anything else, there is a responsibility to the public at large to make society better. In turn, the public recognizes the work and pays dearly for it to be done correctly. That’s what an engineer does.

                                                            No such social contract exists with software developers for the vast majority of modern B2C apps. The users won’t, for whatever reason, pay for the value the software gives them, so of course we’re going to have to get it back somehow.

                                                            That reclamation of value? Through dark patterns and mining and advertisements and everything else? That’s what you’re paid for, at least if you choose to work in a sector where users don’t pay directly for what they get.

                                                            Our job as software engineers is to build things that make the world (or a corner of it) better, things that solve problems. But that’s not our only job. It’s also to be gatekeepers: to prevent ideas that we know are harmful from being realized. What’s the worst that could happen: we get a reputation for giving a damn?

                                                            I am also sick of this attitude. Maybe it’s just the Stockholm syndrome of the startup industry, the general psychopathic zeitgeist of the current business realm, or something else, but I absolutely cannot stand this idea that the developer on the line should be expected to make important decisions with the current hierarchies we subject ourselves to.

                                                            You are not paid to tell the business people that they’re being misanthropic and cynical in their exploitation of users. You are not paid to “make the world a better place”. You are paid to write code that works, on time, on budget, and to spec.

                                                            If you have significant equity in the company (which, let’s be honest here, statistically you don’t), then by all means you should be a standards-bearer for Good in what the company does, at least insofar as you believe it creates a better ROI for your equity. It’ll be a pain in the ass, it’ll be stressful, it’ll be an uphill battle (and you still may lose), but at least you have a dog in the fight.

                                                            If you don’t, though, you shouldn’t bother making a stand as the author suggests. Implement the bare minimum it takes to get the thing out the door and bump the metrics. You don’t have to be extra evil, but there is no reason not to go with the flow. You won’t make more money fighting things, and in fact may make less (because remember, you have no stake in the company’s success, because you have no ownership in it), and all the while you’ll incur extra stress.

                                                            ~

                                                            If you want to be an activist, go write free replacements for platforms and open-source your software and educate users on how their laziness/thriftiness is setting us all up for failure. In the meantime, sit down, shut up, and do your fucking job.

                                                            1. 14

                                                              You are not paid to tell the business people that they’re being misanthropic and cynical in their exploitation of users. You are not paid to “make the world a better place”. You are paid to write code that works, on time, on budget, and to spec.

                                                              Your responsibilities as a person do not begin and end with what you’re paid to do.

                                                              1. 5

                                                                If you take your responsibilities as seriously, you shouldn’t work at places whose business model guarantees eventual convergence to the business methods the article talks about.

                                                                Remember, right from the start of the article:

                                                                This idea is bad because it trades a worse product for a better “business”: revenue, eyeballs, impressions, you know the drill.

                                                                And that’s the thing–if you are voluntarily working for a business, then your job is whatever is best for the business. The fact of the matter is that the companies that make the most money fastest tend to engage directly in the sorts of shenanigans the article decries–Google, Facebook, Palantir, Zynga, Linkedin, etc.

                                                                If you work at a business that competes in those same sectors, the business will eventually have to engage in those same practices or be pushed aside as the less scrupulous companies consolidate their market share.

                                                                Your customers will leave you because the alternative has no direct monetary cost to them, and then you go out of business and the customers get exploited anyways. Customers, because they are people, are not smart about such things.

                                                              2. 7

                                                                I upvoted your rebuttal because it made it clear what you believe and its on-topic. That is appreciated.

                                                                But, I think you’re wrong.

                                                                1. 2

                                                                  That’s quite alright. We can all get along just fine even if we don’t agree. :)

                                                                  Out of curiosity, which bit do you think I’m wrong about?

                                                                2. 11
                                                                  there is no reason not to go with the flow
                                                                  

                                                                  How about being able to sleep at night? Why should owning significant equity be a precondition for moral behavior?

                                                                  You are paid to write code that works, on time, on budget, and to spec.
                                                                  

                                                                  I’m paid to help the business succeed, not to do what I’m told without thinking about it.

                                                                  1. 6

                                                                    How about being able to sleep at night?

                                                                    It’s a hell of a lot easier to sleep when you don’t dread fighting your biz team every following day. If you truly can’t sleep at night, quit the job–don’t bother trying to fix things.

                                                                    I’m paid to help the business succeed, not to do what I’m told without thinking about it.

                                                                    What will help the business succeed is to extract maximum lifetime expected value from its customers, to grow its number of customers, and to finally extract maximum value from its intellectual property as is stops growing. Even more cynically, the success of the business–big picture–is increasingly only defined as obtaining a successful exit for the early shareholders (founders, investors, etc.).

                                                                    In neither of scenarios does the tactic “don’t exploit your customers as far as you can get away with” enter the picture.

                                                                    Again, if this bothers you, go work for a non-profit. or start giving away your work product. We don’t have to keep playing this game and supporting it. If developers actually walked off the job, and users actually were pushed hard enough to really scream vocally, the goals of this activist approach might be reached. In the meantime, though, “activists” like this just continue to contribute to these businesses, albeit less efficiently and at greater cost to themselves than they are otherwise capable of.

                                                                    1. 8

                                                                      don’t bother trying to fix things.

                                                                      An alarmingly defeatist attitude. “The customers still pay for the software even though it has this annoying bug, no need to bother fixing it. Just go with the flow. Do what you’re told.” While we might not want to say it’s a categorical imperative to take a stand against such things, wanting to do so and use your expertise and knowledge to make the business better in addition to more profitable is an entirely valid and reasonable position.

                                                                      What will help the business succeed is to extract maximum lifetime expected value from its customers, to grow its number of customers, and to finally extract maximum value from its intellectual property as is stops growing

                                                                      In neither of scenarios does the tactic “don’t exploit your customers as far as you can get away with” enter the picture.

                                                                      I’d say “don’t exploit your customers” is part and parcel of trying to extract value from them and grow. I donno about you, but I constantly see stuff like this http://users.livejournal.com/joshua_/61105.html aand this https://news.ycombinator.com/item?id=9667809 everywhere I look in this industry, every day. These things are not just bad for the customer. Hurting the customer hurts the business when those customers lose their patience and go somewhere else that doesn’t engage in these sorts of practices.

                                                                      Again, if this bothers you, go work for a non-profit. or start giving away your work product.

                                                                      This is the IT version of the “if you don’t like it in $COUNTRY then move somewhere else!” non-argument. It is possible to contribute to a team/product/company you don’t agree with on every point, and it’s also possible to try and improve these organizations from the outside. Going nuclear on every disagreement by walking out of the room is not how real humans interact to solve problems.

                                                                      Edit to add more rambling things:

                                                                      There have been a few times that I have been asked to do things that I thought were wrong for the business and wrong for the customer. In the vast majority of cases, the business ended up agreeing with me. In the other cases, there were a few that were outrageous enough I simply requested that the project be assigned to someone else. In every single one of those cases, the business was happy to comply and I’ve never been hounded by the biz team afterward. There have certainly been heated arguments, but when the dust settles, the decisions are made, and we move onto the next thing we’ve still been able to work constructively together with no hard feelings. Fighting to improve things doesn’t actually mean everything has to be a fight - in fact that’s almost certainly the wrong way to go about it. You build a good relationship with the rest of the people you work with and when the time comes to make your argument they’ll often listen because you’ve built that trust.

                                                                      Perhaps I’ve been very lucky to work with these sorts of people in these sorts of environments. It’s entirely possible. But it seems to me I’m getting paid to contribute to the business - if I have a strong argument about a business direction decision it’s usually just as well received as a strong argument about a software design decision.

                                                                      I’ve probably been very lucky, actually, and have worked with great people, so I feel pretty strongly that this is generally the right thing to do. One time things got so heated about a business decision that I had a screaming match with the CEO - probably not the best career decision. After we tired ourselves out being ridiculous, we came to a reasonable compromise, shook hands, and went out for a beer after work. Maybe that’s not how most (or even many) companies are, but wouldn’t it be nicer if it were?

                                                                      1. 3

                                                                        Hurting the customer hurts the business when those customers lose their patience and go somewhere else that doesn’t engage in these sorts of practices.

                                                                        Which is why everybody is leaving Google, Facebook, Microsoft, Twitter, and Linkedin, right? In enlightened droves?

                                                                        The sad fact of the matter is that, using VC to undercut traditional revenue models and kill off competitors, modern B2C fast-growth startups can effectively use the thriftiness and shortsightedness of customers against the customers' own best interests. In doing so, they also salt the earth for any more reasonable business model.

                                                                        Like, can we just stop pretending, in certain industries, that it is anything other than optional to even give the barest respect to your customers?

                                                                        They’ll lap up whatever is put in front of them, seeking the littlest dopamine rush from new and shiny and free-because-dark-practices-funded. This is literally how Buzzfeed, Cheeseburger, Facebook, and all that work.

                                                                        You know why it’s called a feed? Because the customers are increasingly dumb livestock.

                                                                        ~

                                                                        Look, I’m not saying this because I hate users. I’m not saying this because I despise growth-centric B2C and social plays. Don’t get pissed at me for that.

                                                                        We all want to live in a world where developers help users get the most out of life. We all want to live in a world where users are fully aware of what they’re getting themselves into, where companies are totally upfront about how they’ll treat and monetize their users, and where the bad actors are punished for acting badly.

                                                                        We do not yet live in that world, and trying to act otherwise is just setting ourselves and our users up for disaster.

                                                                        I’m saying these things because we can’t fix anything until we acknowledge deep down in our bones and heart of hearts that many of these ecosystems are deeply fucked in the head and the soul and, more importantly, that they have no incentive to ever act differently when developers are content to do anything but leave.

                                                                        1. 2

                                                                          All true, but, just because dev X recognises that things are fucked, does dev X have to contribute to the endless perpetuation of its fuckedness?

                                                                          I agree totally that the bullshit is pervasive and seemingly ubiquitous and that we probably have a 50-200 year battle to break it down. It’s fucked, and it’s going to be a long and thankless and more often than not seemingly impossible task because the barriers have been so long and so effectively established. But… [a] we don’t have to totally collude with it, even maintaining some form of internal distinction between “what’s good” vs “how shit is” holds value for the world, even if it’s just in keeping some flame of meaning and humanity alive; [b] just giving in and saying “fuck it, I’ll follow orders and not even question them” is kind of an issue with humans, and [c] dude, you sound like you need a vacation. I know you not a jot but I hear a lot of these arguments in my own head with some of the same conclusions and seriously, please, don’t give up. If smart people like you give up, if the intellectuals and the artists and the creators just give up and submit to the tsunami of shit eating the world, then we really are all fucked.

                                                                  2. 5

                                                                    I see your answer as a providing excuses for make unethical choices. Basically you say “in the domain of web stuff, there is less money to be made if you refuse harming your users”. Is it not possible to make enough money without harming your users? Is it not possible to choose to work in a difference setting? You’re making it look like people are forced to work on harmful parts of web applications because “this is what they are paid for”, but we choose what we are paid for.

                                                                    There are plenty of situations where people in practice do not have much choice of which life to live, which work to be paid for. Programmers have highly sought-after skills that can guarantee a job all over the world. We have lots of choices of what to do and by whom to be paid. If deciding to “go with the flow” makes you develop unethical programs that do harm, it is your choice that enables this harm.

                                                                    1. 2

                                                                      I see @angersock’s answer more as a question of whether you’re doing more dedicating minimum effort to that work whose goals cannot align with what the developer thinks is right, while working on projects whose goals are aligned with what they see as good, instead of trying to swim upstream their whole life, and exhausting themselves for no reason; not whether the said developers should simply give in to the lethargy of the dissociative, dissonant world of profit margins. The needs and actions of a machine ran off capital will always be irrational in at least some way, since profit rarely means good for anyone but the person receiving it, especially as tangibility of those exchanges descends into the realms of attention and abstracts.

                                                                      There’s no escaping that for a lot of people, I think. Not everyone gets a choice in going to work where elusive “noble profit” exists.

                                                                    2. 2

                                                                      In the meantime, sit down, shut up, and do your fucking job.

                                                                      Not fond of Snowden, either, eh? Or other objectors to authority?

                                                                      1. 5

                                                                        Huh? No, Snowden was fine–least of all because he deliberately compromised the organization he worked for. He did not try to save it, he did not try to make it less evil, he did not fight a hopeless battle, and he seemed to have done it knowing full well what the personal cost to it was.

                                                                        He did so when the stakes were far greater than himself: the freedom of the populace at large, not just a few extra bucks difference in monthly revenue.

                                                                        He also worked for a .gov–that’s a somewhat different calculus entirely than what the author and I are considering here.

                                                                      2. 2

                                                                        Your point is somewhat absolute, I can agree with a less absolute point.

                                                                        As a team member I think you need to speak up if you think something is amiss. If some one in charge of the roadmap dismisses the idea, unless it is a foundational problem (like immoral or illegal) I agree you should not fight it if it is not explicitly your job to challenge that idea. The basic assumption there - often wrong - is that the first party is competent and has access to more information/wisdom than you.

                                                                        However: If there are many such events and they often are foundational points for you, I suspect it will soon be time to vote with your feet and move to a different organization, since it seems that the core values of the organization (“culture”) clash with your own.

                                                                        1. 1

                                                                          That reclamation of value? Through dark patterns and mining and advertisements and everything else? That’s what you’re paid for, at least if you choose to work in a sector where users don’t pay directly for what they get.

                                                                          I can’t believe anyone would upvote you for advocating dark patterns. Disgusting. That is the same reasoning that drives all of the filth we receive in our Inboxes on a daily basis, and the various decisions that result in malware compromising people’s computers and generally making people’s lives miserable for the sake of your profit. Pure evil. Please go peddle this junk somewhere else, not on Lobsters.

                                                                          1. 2

                                                                            You need to read more carefully, my friend.

                                                                            I’m not advocating dark patterns in general, and frankly I find them quite abhorrent. I’ve seen them ravage one industry (games), and it’s been annoying watching the same thing happen elsewhere. I would appreciate it if you don’t put words in my mouth.

                                                                            The point is very much in the quote you yourself picked: “at least if you choose to work in a sector where users don’t pay directly for what they get”.

                                                                            My answer to this conundrum was to either move to a different sector, or to go and give away the software that solves the problem–because the exploitation of the users is built into the successful business strategies for those sectors.

                                                                            Evil is profiting off of the misery of others.

                                                                            This statement you linked to is pithy, but ambiguous. It is unclear whether you mean profit from the creation, maintenance, or cessation of misery is evil.

                                                                            1. 1

                                                                              I’m not advocating dark patterns in general, and frankly I find them quite abhorrent.

                                                                              Good to hear that, but I hope you can see how your statement could be interpreted as though you were.

                                                                              This statement you linked to is pithy, but ambiguous. It is unclear whether you mean profit from the creation, maintenance, or cessation of misery is evil.

                                                                              The first two (creating and/or sustaining misery for the sake of profit). The latter is a good thing. Please do profit from the cessation of suffering, I will send you kudos for doing so. Just realize that you’ll be selling cures, and selling cures means you will be destroying your own business as you go, so you’ll have to continually find new things to cure.

                                                                        1. 5

                                                                          Emacs: Don’t bother. Just say forever and bask in the emacs-y goodness.

                                                                          1. 1

                                                                            They forgot about M-x kill-emacs. Tsk tsk!

                                                                            1. 1

                                                                              Indeed. I recently disabled C-x C-c, because as my experience has grown I want it running but the muscle memory for “make window go away [by quitting]” was too hard to overcome. Now I still type it, but I don’t lose my open buffers :-)

                                                                          1. 2

                                                                            I’ve been interested in human factors in complex systems lately so I’ve been reading Three Mile Island: A Report To The Commissioners And To The Public, Vol.I and Normal Accidents: Living with High Risk Technologies by Charles Perrow.

                                                                            For relaxing spare-time reading currently I’ve got The Knight and Knave of Swords by Fritz Leiber, because I enjoyed the series when I was a kid but my dad’s collection ended at Swords Against Wizardry, and I only recently discovered that there was more.