1. 69

Just started work on this idea: really solid resources that act as cold showers on overhyped programming topics. I’d love it if you all sent in your favorites!

  1.  

  2. 17

    I would love to see something like this for “Serverless” and lambda-style architecture. I believe these are hyped but have some serious downsides.

    1. 3

      I’ve found the criticisms levied at the Actor Model model apply to the serverless architecture. AWS / Google’s offerings link their existing services together to feed a mailbox that your function gets invoked by.

      1. 1

        ….why does that website not display without javascript enabled? it makes no sense to me…

        1. 1

          Its been converted to a single page app.

      2. 1

        I was curious myself since I haven’t looked at it in a while. It was kind of hard to get results since searching for its issues actually brought up more pro-serverless articles than anti-serverless. My brief Googling found a Not Ready for Primetime article in February 2016, these issues in June 2016, a Challenges of Serverless article in March 2017, a few drawbacks in May 2017, and a Quora answer with a few more gripes.

      3. 13

        I don’t have any research papers off hand, but I would love to read some criticisms of the blockchain/cryptocurrencies (in terms of technology rather than economics).

        1. 6

          This piece is pretty good.

          1. 13

            No, that piece is actually uniquely terrible. I mean, if you want to simply read a piece which throws cold water then sure, read it, but you’re probably looking for informed pieces. The linked article is written by somebody who doesn’t understand the space; it makes claims which can be trivially disproved by reading the relevant literature. (For example, he seems to think that systems like Filecoin are proposing we store bulk data directly onto a blockchain)

            This one is much better.

            Attack of the 50 foot blockchain, both the book and the blog, are also great. They go less into technical details and more into pointing out the crazy claims some people are making and the crazy schemes they’re proposing.

            1. 6

              This one is much better.

              That looks like it might be worth including on the list. I’ll try to review it tonight.

              1. 6

                I thought it was a pretty good piece. Main factual errors are: eight years not ten, and bitcoin transactions haven’t been four cents since early 2015.

                But I think his central thesis is sound: all of this is a series of very impressive solutions to things that aren’t in fact the problem. Key quote:

                In conversations with bitcoin entrepreneurs and investors and consultants, there was often a lack of knowledge or even interest in how the jobs were being done today or what the value to the end user was.

                The more general problem is that “blockchain” the buzzword has less and less to do with the weird trick Satoshi Nakamoto came up with. e.g. Hey, Estonia’s all Blockchain now! Everything there is on blockchains! Is this Good News for Bitcoin?? well, not quite, because their esteemed Guardtime KSI Blockchain isn’t actually, er, a blockchain. That sort of thing.

                1. 2

                  Oh, it’s cool that you’re here. First off, thanks for the book, it gave me a very lovely Saturday!

                  In conversations with bitcoin entrepreneurs and investors and consultants, there was often a lack of knowledge or even interest in how the jobs were being done today or what the value to the end user was.

                  If this was his main point I would agree that it was a decent piece, but all of his examples really are terrible. He focuses on b2c applications; almost all of his reasons are how consumers already have much better alternatives than anything blockchain-powered currently provides.

                  I agree, however (1) there’s much more to the world than just b2c, (2) b2c is usually built as a wrapper on top of unfriendly systems. Dropbox, one of his examples, is built on top of s3, which isn’t much friendlier than Filecoin would be, if it succeeds. Filecoin has the promise of being much cheaper though, so you can imagine, 5 years from now, using a Filecoin-powered-Dropbox which gives you all the same features but is less than half the cost. I know that doesn’t sound revolutionary, but before I lose your interest by claiming even bigger things I wonder if you agree the above is possible.

                  He then prematurely generalizes, and claims that these failings prove the entire class of blockchain protocols are doomed. Again, I totally agree that consumers prefer airline miles to whatever libertarians mean by “sound money”. However, I think things like storage markets, prediction markets, incentivized mesh networking… all of these take time to build; and the author of this piece seems to think that because he can’t imagine them then they must surely not exist.

                  The more general problem is that “blockchain” the buzzword has less and less to do with the weird trick Satoshi Nakamoto came up with.

                  This is, in some circles, completely true. It’s something Matt Levine has written about; blockchain is often code for “let’s get decision makers to approve fixing some of our infrastructure by giving the work a sexy name”. You’re reading your own opinions into his piece though, I don’t think he ever argues this :)

                  1. 6

                    I probably am reading my own opinions into his paper to a degree, then agreeing with myself, yes :-) I was reading it as a businessperson’s frustration with fountains of BS that never quite work out.

                    I do think, though, that eight years in, the burden of proof is firmly on the blockchain promoters. I have no patience any more for purported blockchain systems that don’t exist yet, because their track record of achieving existence is really bad.

                    Hypotheticals in the blockchain space are so cheap as to be worthless. “Could” or “will” are words meaning “doesn’t”.

                    The bitcoin press don’t really help in this. CoinDesk in particular never seems to have seen a hypothetical it doesn’t like; all you need is to make your thing slightly blockchainy and they’ll post a piece of journalistic stenography about it. Its failure to eventuate six months later will never be noted. This is a big problem because the mainstream media assume good faith, that this is specialist press rather than boosterism press …

                    Try this yourself! Take any of these hype-filled proposals and replace “could” with “doesn’t”. I’ve got here a blockchain in healthcare paper - peer-reviewed!! though in a journal that made Beall’s List - that has “could” 81 times, all for things that DON’T EXIST. Perhaps one day they will! But the fact that they DON’T EXIST is in fact an important point.

                    The Lightning Network is one personal bugbear on this point. I have had this argument, live on video with a bitcoin advocate (they didn’t put that bit up, for some reason … though I still have my video) where I pointed out bitcoin’s utter failure to scale and got back “but look, the Lightning Network! (laugh)” as if that was a slam-dunk refutation. I had to point out the fact that LN’s been coming any moment now in 2015, 2016, 2017 and for 2018, and the most important thing about it is that it DOESN’T EXIST. And, y’know, its repeated failure to happen might be relevant to its value as a solution to the problem.

                    (Rant time: I just had another go-round recently with LN advocates. I am pleased to say the actual devs are much better to talk to than the advocates. There is now LN code! The crippling flaw of the whole LN design remains the path-finding algorithm, to go from arbitrary node A to arbitrary node B - which is the same problem as trying to solve a UUCP bang-path. What was the best that the most motivated computer scientists and sysadmins of the 1980s could manage? Either compute it by hand, or download monthly maps of the entire network and pathalias it. And they all ran screaming for DNS as soon as they had proper Internet. So to do a proper mesh network, rather than just a few hubs closely resembling the Visa-Mastercard-PayPal triumvirate, LN actually requires new computer science. The present alpha code uses “broadcast every channel and transaction to the whole network”, which is I think O(n^2) and scales to literally tens of nodes. And then, even if it all worked, the economics of the LN can’t possibly work and would seize up immediately. But as far as I can tell, they literally don’t have a planned or anticipated economic model; certainly there’s nothing in years of dev list messages referencing any sort of thing they’re aiming for, which any reasonable person would think was something that might be needed … gah. Hyped vaporware. I tire of it.)

                    so yeah. I have no patience for hypotheticals in the blockchain space any more. Call me when the thing exists and works. There’s too much BS and too many BSers.

                    I did just ask an actual mathematician working in related areas to write a section-by-section takedown of that TimeCube-in-LaTeX white paper IOTA put up. I hope they have something I can link or post soon.

                    1. 2

                      I do think, though, that eight years in, the burden of proof is firmly on the blockchain promoters. I have no patience any more for purported blockchain systems that don’t exist yet, because their track record of achieving existence is really bad.

                      I really can’t argue. I do think that these things take time. Many of the potential “could” and “will” applications assume things like low transaction fees or a lightning network or decentralized storage. Until we figure out how to scale these things those applications are likely to be out of reach.

                      I’ll have to begrudgingly agree with you if we go another 8 years without any big breakthroughs but for now I’m hopeful, ethresear.ch has some cool ideas which are just crazy enough to work. It took a surprisingly long time for the steam engine to become practical.

                      And then, even if it all worked, the economics of the LN can’t possibly work and would seize up immediately

                      Do you have any references I could go read? I have my doubts on Lightning Network, it feels like there need to be hubs, and the channels between those hubs need to be massive to hold all the payment volume, but that’s just my idle speculation. I’d love to see some math.

                      I did just ask an actual mathematician working in related areas to write a section-by-section takedown of that TimeCube-in-LaTeX white paper IOTA put up. I hope they have something I can link or post soon.

                      Yes, IOTA is an unmitigated disaster. I also would love to have something I can link to.

                      1. 3

                        No, I was trying to put one together, but that really requires the LN to be an actual thing that could be talked about. It’s hard to make specific and pointed critiques of a nonexistent network. I mean, proponents can easily say “that’s not a problem because (spurious justification)” because the network doesn’t exist yet

                        Though here’s someone writing several obvious arguments against it. Also, Jorge Stolfi on Reddit (/u/jstolfi) has critiqued it at length and in detail.

                        Whenever I’ve seen someone raise economic arguments, LN devs and proponents go silent. I’m also pretty sure you can get credit - the promise of money that would be as usable as money - out of it, though the notion is anathema to bitcoiners because credit is evil therefore doesn’t exist. Etc., and on.

                        The LN is very un-blockchain in nature. One of the big things, actually, that worries me about the LN is that I don’t think there’s been a system like it in financial history. And people have tried just about everything, so that’s actually a point against it. I asked one finance history buff, and the only thing they could think of was Bretton-Woods and large quantities of gold. And B-W eventually faded away as people stopped pretending they cared about the actual gold.

                        I’m seriously starting to think they were desperate for something, anything to deal with the Bitcoin transaction clog and seized upon this half-formed LN payment channels idea, didn’t think it through at all, and still haven’t.

                        The LN on the testnet is pretty elaborate. Buuuut the testnet doesn’t have $50 transaction fees to set up a channel.

                        1. 2

                          Thanks for making your points in this thread. They’re well-stated and express some of the concerns I’ve had while evaluating various possibilities for the current project I’m working on, better than I could have.

                2. 3

                  The first is pretty good esp on speed, flexibility, and energy use. If it’s on the list, one might want to include prior and alternative work that uses traditional tech with audit ledgers, byzantine databases, securities built on multiple stable commodities/currencies, nonprofits solving for-profit transaction issues via charter + contracts, and so on. Lots of possibilities that reuse everything from proven security or software methods to well-understood laws and business models.

                  Strange enough, echo chamber is so strong on bitcoin that anyone bringing up stuff like that just fixing centralized methods or decentralizing without blockchains gets pushed to bottom of thread. Meanwhile, the newly popular thing is still failing while the traditional methods still mostly work.

                  1. 3

                    Fairly sure David Gerard is a fellow crustacean. (He and I seem to end up in all the same internet places for some reason.)

                  2. 3

                    That’s more an argument about the business side than the core technology, though.

                3. 15

                  Great idea! For formal methods, I thought this paper was a great start on criteria for deciding which, if any, formal methods to adopt in an industrial project. It shows strengths and weaknesses of a bunch of them with the best picks matching what experience reports Ive read pretty well.

                  1. 10

                    Wow, inlining the abstract:

                    Despite extensive evangelizing and demonstration of several success stories in safety-critical applications, formal methods are still not widely practiced in contemporary systems and software engineering. One of the main reasons for this situation is the absence of systematic guidelines and evaluation criteria that help software practitioners choose the right formal method for the problem at hand. In this article, we present a comprehensive set of criteria for evaluating and comparing different formal methods, based on a systematic literature review and decade-long personal experience with the application of formal methods in industrial projects. We argue that besides technical grounds (e.g., modeling capabilities and supported development phases), formal methods should also be evaluated from social and industrial perspectives. We also evaluate several state-based formal methods on the stipulated criteria.

                    This looks great!

                  2. 6

                    Along the lines of The Leprechauns of Software Engineering it seems, very good.

                    I especially like the static languages thing, a point that keeps getting repeated but in my experience even in dynamic langs (JS) I never had much trouble that types would’ve avoided.

                    1. 6

                      I thought this was actually going to be about cold showers, which in fact do have a lot of awesome health benefits.

                      1. 8

                        The literal interpretation given the tags was @hwayne was going to tell us the benefits of cold showers, the culture that’s developed around taking them, show his proposal on how to do it better, and ask for feedback from Lobsters brave enough to jump in one.

                        Presumably with a PlusCal spec of states ranging from SteppingIn to JumpingOutWithProfanity. Probably a two, state spec as I cant imagine much middle ground.

                        1. 1

                          We definitely need a category to add this one if you find any papers on it.

                        2. 4

                          The one about big data vs. the laptop made me laugh and cry all at the same time.

                          1. 6

                            This isn’t a research paper, but the same idea:

                            https://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html

                            I have seen this happen in real life too… I think I mentioned somewhere that learning bash is harder than learning Java or C++ for some people, hence you get these horribly inefficient solutions.

                            1. 1

                              Things that aren’t research papers are 100% welcome, too. In fact I’m worried about research papers being overrepresented, since they’re so much easier to find.

                            2. 2

                              It was all laughter and WTH’s when I saw it after reading all the articles about the clustered tools. I about couldnt believe how far marketing had taken them.

                            3. 4

                              This reminds me of a neat lecture by Greg Wilson about this type of hype killing research: Greg Wilson - What We Actually Know About Software Development, and Why We Believe It’s True.

                              1. 5

                                Along the same lines, his book Making Software is great also.

                              2. 4

                                One of the many things I love about http://learnyousomeerlang.com/ is that he ( @ferd ) regularly includes “don’t drink too much kool-aid” comments.

                                1. 4

                                  It might be a bit old news, but finding some for microservices probably shouldn’t be too hard to find.

                                  https://aadrake.com/posts/2017-05-20-enough-with-the-microservices.html

                                  1. 4

                                    I think a lot about how we handle evidence and initiatives like this. The inconclusivity we have on issues like dynamic typing and formal methods is sort of like the replication crisis in the social sciences. There’s the question of what to do next once we’ve stepped out of the cold shower - especially when we take into account the way that the mind works.

                                    I can’t find a link right now, but cognitive science seems to show that people give higher weight to the first piece of information they hear about a subject regardless of later disconfirming evidence. So, for example, if you are in a jury and the judge tells you to disregard the last testimony you heard, it doesn’t really work. Because of this, what we generally do is consciously override that weight and become contrarian or worse non-committal because there’s evidence for and against everything.

                                    I think it would be great if most of software development borrowed the framing that the software patterns movement used - all choices in software are design choices. We have forces and a context. Whenever we make a choice there are both positive and negative consequences. We should be able to enumerate the positive and negative for everything and also be aware of opportunity cost. When we have these sorts of discussion about whether a technique or technology is “good” or “bad” it’s context free and it really shouldn’t be.

                                    1. 4

                                      This isn’t specific to programming, but Why Most Published Research Findings Are False [1] throws a lot of cold water on scientific studies involving p-values. Computer science doesn’t use p-values as much as other sciences, but they are used.

                                      I highly recommend the NYTimes article When the Revolution Came for Amy Cuddy [2] for an in-depth look at what has happened with p-values.

                                      This researcher gave the most highly watched TED talk [3] of all time (or maybe one of them.) And then the entire field changed its standards and she was abandoned and humiliated by her colleagues.

                                      Nobel prize winner and popular author Daniel Kahneman also admits “I placed too much faith in underpowered studies”. In other words, many of the things he wrote about in his books are false.

                                      Also, it is tempting to be smug about “hard vs soft” sciences. But I see no indication that your average computer scientist will do better than your average psychologist. They still use p-values of 0.05, and (from experience) the experimental hygiene and reproducibility is poor. If anything, computer science should have the best reproducibility, because everything is done on the computer!!! But we don’t, which is a shame.

                                      [1] http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

                                      https://en.wikipedia.org/wiki/John_Ioannidis

                                      [2] https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html

                                      [3] https://www.ted.com/talks/amy_cuddy_your_body_language_shapes_who_you_are