1. 34

  2. [Comment removed by author]

    1. 12

      I’ve yet to be convinced that quality sells. In the same vein, testing doesn’t sell either.

        1. 1

          Never heard of that before, but seems pretty accurate :(

        2. 5

          A problem I see in most of these conversations is squishy and variable definitions of “quality”. Here’s the definition I like: ((net present value of customer satisfaction) / cost). Customer is whoever the beneficiary is, discount rate varies by situation: high for a prototype at an early stage startup, low if you’re building firmware for a hydroelectric turbine controller that’s going to be in service for decades. Everything you do from choice of tools to management style can be evaluated by its contribution to quality.

          I’ve written some fragile, buggy code that had to ship in a matter of hours but it was a high quality product: it did what the client needed when they needed it. Fewer bugs and a day later would have been a useless low quality solution. I’ve also spent weeks on a pretty tiny networked data synchronization service to ensure that the internal logic was solid and that the error handling was thorough (which on cheap Android devices is a project). That code was reliable in the field for a long time, high quality. On another project I wrote some transaction processing code in Python (due to the team’s existing language familiarity) with mypy for type checking and Hypothesis as part of the testing framework. The code was clean and in writing it we actually found bugs in the old code but ultimately the tooling was beyond the team’s comfort. Not high quality.

          This victim mentality us-vs-them stuff is poisonous and pointless. We’re highly skilled professionals in a highly sought-after field, we have the leverage to push change inside companies and the option to vote with our feet. There’s no excuse for claiming everyone else is “doing it wrong” and not doing something about it. When you do act the test of rightness is easy: how good of a job you do is ultimately measured by customer satisfaction. If you make something good for the customer at a reasonable cost and go home happy, that’s quality work. It’s not complicated.

          Edit: I’ve worked with a lot of people at a lot of places. I don’t get this idea that there are a huge number of stupid and/or malicious people in the industry, I haven’t seen it and I don’t think that view reflects reality. There are lots of problems but that’s not one.

        3. 11

          This is blame shifting. Is management to blame for a lot of buggy software due to unreasonable deadlines, lack of funding for QA, lack of budget for hiring teams of an appropriate size with skilled developers, etc…? Absolutely.

          There’s also tons of open-source software out there which has no such stereotypical PHB behind it. The release schedule is dictated by community consensus or the relevant senior developers. And this software is often just as horrible as the output of so many closed-source behemoths we all love to hate.

          Management always gets a bad rap, and a lot of it is deserved, but they can’t always be the foil for bad products. Especially where management largely doesn’t even exist, or does so in a far less formalised sense per many community projects.

          1. [Comment removed by author]

            1. 2

              good management is really hard, and should be treasured above all

              The problem with executives is that no one is willing to admit that 90% of them are negative-net-productivity players.

              Sure, I agree that the best 2% of executives are worth millions per year to their respective companies. Even though Steve Jobs was an asshole, he was a competent asshole which enabled him to clear out the incompetent assholes who had managed to accumulate in Apple’s ranks. A CEO that can cut through bullshit and cut bikeshedding petty tyrants down to size can save a company. (Of course, such CEOs are rare. Smiling, affable dipshits are more likely to get that job.)

              The problem, in the executive suite, is that the garbage also gets the high pay, status, and control. No one is willing to admit that most of these people are worthless and got their jobs through nepotism. There’s also nothing in place to filter them out, either, because managerial power gives them the ability to extort people into supporting their own careers.

            2. 3

              Programmers and technology management are both to blame. You can’t separate the two groups into separate tribes, especially since most of these bad middle managers that people love to complain about (and, in truth, they deserve their bad reputation) are ex-programmers who went Uncle Tom; as soon as they got that 20% raise, they were all about . I’ve encountered plenty of talented engineers who turned into awful managers.

              I also think that corporate life makes people stupid, unless they have the courage to fight it, and then they’re putting their careers at risk. The supposed loss of “fluid intelligence” that happens in middle age, in my opinion, has no biological basis. I think that it’s an epidemic of subclinical midlife depression (not enough to put a person on meds, but enough to cause a 1-IQ-point-per-year drop over 30 years) that the militant mediocrity/anti-intellectual corporate world has created in people. Thus, you have people falling to pieces at a time when they’d otherwise be at their prime (the few who are lucky enough to dedicate their lives to meaningful intellectual pursuits tend to peak around 55, not at 23).

              In other words, the militant stupidity that’s enforced upon us with open-plan offices and Scrum doesn’t magically go away when people go home and moonlight on open-source projects.

            3. 6

              I think it’s a bit of both. There’s definitely more pressure to get stuff out than before but I also think sometimes devs get lazy or just have poor attention to detail. Once you’re not proud of the product you work on, it’s easier to aim for “good enough” than “great” :/

              1. 4

                This. It’s hard to give software enough care when you need to deliver, deliver, deliver…

                You end up taking shortcuts in the end. Saying to yourself that you will refactor it later on. Instead a new project comes and you leave the ugly mess behind, hoping the new one is finally going to be the nice one. Except you need to deliver…

              2. 3

                And if a last ditch defense of that position were ever needed, it’s easy to show that there are good developers out there who will do quality development given half a chance, and the managers either don’t hire them or don’t give them that half a chance.

                The culture is hostile to people who care about quality. You can’t think in these open-plan offices that are designed to inculcate the idea that you matter more for your availability and appearance (especially to investors) than your actual capability. (An open-plan programmer is more valued as office furniture, to create a “busy” impression when investors are on-site, than for the software she creates.) Moreover, Scrum (i.e., terrorism for people who aren’t technical enough to make bombs) has been repurposed as a micromanagement framework designed to humiliate perceived low performers, with the tolerated side effect of also hitting people who actually care about their work well enough to do it right.

                Thanks to the Silicon Valley’s willingness to bring absolute human garbage into the Founder class, private sector dev culture has become low-status, humiliating, and wracked by the crab mentality. It’s also sexist, racist, classist, and ageist and shows no signs of improving. In this light, it’s not surprising that low software quality would be the norm. This is a world in which everything’s built to be sold and 99 percent of it is garbage. Most people have lost the ability to create solid assets; it’s been punished out of them (again, Scrum).

                However, I don’t think that it can be blamed only on software management. I understand the self-serving tribalism. It’s intellectually easier to heap all the blame and hatred on some Other called tech management and, sure, most technology executives are horrible people. That said, I don’t think that we can put 100% of the blame for our shitty culture and terrible products on “Them”. We’re at fault, too. I’ve seen engineers (even talented ones) become the worst kinds of people when given a little power. Also, whose fault is it that we’ve failed to organize around our interests? It’s ours. If we really care about software quality, then why aren’t we forming collective structures that can kill Scrum and the open-plan office culture?

                There are some wonderful people in technology, but I’ve been in this game for 10 years, and all I’ve seen is that the bad people drive out the good. It exists at all levels and it can’t be blamed on management only. Those awful managers wouldn’t be in charge if engineers weren’t so easy to divide against each other (tabs versus spaces, Ruby versus Python, California versus “flyover country”, young versus “old” (meaning 30+), women versus brogrammers, and so on).

              3. 10

                It’s an understatement to say that this resonates with me. Yesterday was frankly a shit day for me, and about half of it was a write-off, precisely because I was dealing with software bugs. When you encounter a rapid series of them it’s incredibly disruptive to productivity and more than a little demoralising. Here’s a select sample of some of what I saw:

                • My primary Linux development VM completely died with a thoroughly corrupted ext4 filesystem during a dpkg update. The root filesystem went read-only with “deleted inode referenced” messages in the kernel log. Rebooted it after a little poking around and it won’t even get past GRUB Rescue. Appears to be a corrupt superblock and who know’s what else. Unsure if it’s the hypervisor (VMware) or the kernel to blame. Either is entirely plausible.
                • Several hangs in Remote Desktop Manager while exiting RDP sessions. I’ve dealt with this sporadically for well over a year and finally decided to spend some time trying to debug it. Appears to be a race condition between two threads in the RDP ActiveX control it embeds. It’s probably more likely to be a bug in the .NET Interop assemblies.
                • Updating a finance application would hang during the MSI update. Originally thought it was the finance app, but later found out it appears to be a thread in one of the MSI processes making WMI calls crashing. Today I found out that WMI itself seemed to be in an unstable state thanks to callbacks registered by an Intel service that’s part of the Ethernet drivers that wasn’t responding to them. Essentially WMI calls were getting stuck on RPC calls that would never complete. This disaster goes deeper, but suffice to say, just getting to this point took wait chain analysis in a debugger across several processes. I will say that my experience with Intel software is that it’s a digital dumpster fire.
                • Crash in WinDbg thanks to a buffer overflow in an extension DLL (irony not lost on me)
                • Crash in Outlook somewhere in the UIAutomationCore library while writing an email
                • Crash in Word somewhere in the WWLib library

                This is one day, and not even a complete recount. It’s not just closed-source stuff, open-source stuff is often equally terrible. I’ve spent this morning rebuilding my VM post it’s total destruction yesterday, and while these days I’m more skeptical of VMware, it wouldn’t be the first filesystem I’ve lost due to rubbish, poorly tested code in the Linux kernel.

                Most people don’t have the skills to debug buggy applications. It takes time and is frequently very difficult. Particularly when it’s not your application. If you do try and get to the bottom of the issue, prepare more often than not to enter a rabbit hole that would make Alice herself cry. Yes, this is a rant, but for those of us who try to maintain stable systems and do the same for all of those systems we manage (or develop software for), the current state of software quality is frankly appalling.

                1. 6

                  My impression is that Linux quality has fallen a lot in recent years. Since switching to FreeBSD I’ve been much happier and my purely subjective impression is that it’s better/more reliable code.

                  1. 5

                    Reading your post reminded me of the time I lost most of a day trying to figure out why our application was entirely broken a few days after I’d upgraded a component, despite me thoroughly testing it.

                    It turned out to be a bug in only the paid-for version of PostSharp, which you “got for free” on a Monday:


                    It took a lot of people telling them how stupid this was on twitter before they finally agreed to change how this works to avoid inconsistent builds across different days :(

                    1. 1

                      Addendum: For any SysAdmins here, consider configuring all of your client systems to report crash events. On Windows this is fairly easy: setup a Windows Event Collector on a suitable server and deploy subscriptions to clients to push “Application Crash” events. Check the pulled events now and then, or on small networks, maybe even email on each new crash event.

                      It’s frequently insightful, genuinely useful, and horrifying. You’ll also find way more crashes than you likely realised, as you’ll see all the crash events for background processes (services, etc…) that won’t typically present a crash dialogue to the user.

                    2. [Comment removed by author]

                      1. 3

                        Linux is particularly bad; I had may more examples I didn’t include. I recently bought a Pi and have been learning things, and there’s just so much crap you need to “just know” to make things work. For ex. there’s a nice package manager that can do dependencies on Debian (apt), yet many packages install without requesting them and then fail telling you that you need some dependency at runtime! :(

                        This blog post is basically what my twitter timeline has been for the last few months :(

                      2. 4

                        I don’t buy it. This is the same complaint that has been made for decades, and it amounts to the same rosy view of the past that we all end up acquiring.

                        Software was never good. It was always bad.

                        Nobody has given up. People still want to build quality software. Externalities make that hard, as does lack of planning and skill.

                        There is more software now, so that may, perhaps, make it harder to ignore that software is bad? When you see BSODs on ATMs, that’s hard to ignore.

                        But software has always been bad.

                        1. 1

                          But software has always been bad.

                          So maybe it’s just that there’s more of it now? Surely you would agree that the demand has shot up, and the idea that the production of quality software has been able to keep up (even at the same low percentage of the total software produced seen in the past) is quite suspect.

                        2. 3

                          I’ve observed something on my side projects: the easier it is to ship (read: deploy), the more bugs that make it out.

                          Now, this could the fact that my usual habits of “let the compiler catch it” don’t work with dynamic languages. But, still. If I find a bug I’m pretty cavalier about just fixing it and hitting the button again. And possibly making the same mistake!

                          Part of it is how time is siphoned off for side projects; I tend to be a bit of a miser for how long stuff should take, as I slot time for improvements and such. But, I feel like I’m falling into the trap of mistaking going fast for improvement. And I see it happening everywhere.

                          1. 1

                            If you get paid the same to write half assed shitty code that takes a fraction of the time to write as good code, what are people going to do? Managers don’t know what good code looks like, they just know that one person will finish a project faster than another. There is no incentive in this sort of market to do things “correctly”. It can be incredibly frustrating knowing that some websites were written by someone who was paid money to create it.