1. Do you use source control?
-2. Can you make a build in one step?
+2. Can you build and deploy your software in one step?
-3. Do you make daily builds?
+3. Do you build on every commit?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
-6. Do have an up-to-date schedule?
+6. Do you measure your progress in terms of value delivered?
-7. Do you have a spec?
+7. Do you have a runnable spec?
-8. Do programmers have quiet working conditions?
+8. Does your environment foster collaboration?
9. Do you use the best tools money can buy?
-10. Do you have testers?
+10. Is testing everyone's responsibility?
11. Do new candidates write code during their interview?
12. Do you do hallway usability testing?
The change to (7) seems to misunderstand what Joel originally meant by spec, which is a more thorough formal communication than at least the vast majority of automated specs or “BDD” implementations I’ve seen in the wild.
The changes to (6), (8) and (10) seem to be direct disagreements with Joel.
I see your point here, and I think I could have worded it better.
I’ve worked with (what I believe to be) an excellent testing group, however I stand by my claim that testing should be part of everyone’s role. That’s more of what I was thinking, and maybe that’s how I should have worded that one.
While I agree, I do think that having programmers write tests for their code is essential. Our current arrangement:
Developer delivers feature to QA with specs (of the runnable variety)
QA finds crazy bugs, the way that only QA people can.
Developer duplicates and creates a regression spec, GOTO 1.
QA is invaluable in my opinion, especially for large systems with a good deal of ins and outs. In my experience, the developer (i.e. me) is not always able to see the edge cases. At the same time, having regressions and specs and programmers writing their own tests also ensures that QA isn’t finding the same bug repeatedly.
A good QA group can definitely be beneficial, but they need to be supplemental not the main test effort. It’s easy to make everyone responsible for testing with policy. Landing new code without a functional test? Fixing a bug without a regression test? Your patch doesn’t get past review.
Their job is to provide evidence that the quality of the product is understood, and to predict how the product will perform when it is deployed, so that the business can make an informed decision on when to release product. Their performance is measured by the strength of that evidence and the accuracy of that prediction.
As such, a QA team is responsible for checking that software matches its requirements, and that those requirements make sense. The QA team provides oversight for both the development and product management teams, and is the third side of the DEV-PM-QA product development structure.
QA is NOT the same thing as “making certain the product works”- that would be Quality Ensurance. If what you have is a group of engineers who are measured by defects found over time, then you have a QE team. You might still ship software, but your developers and testers are likely at odds with each other all the time. Do QA and Dev hate each other? You’ve probably confused QA with QE.
QA is certainly NOT there to provide a group of scapegoats to take the blame when bugs happen in the field- that would be Quality Insurance. If you think your QA / test people are somehow lower on the ‘old totem pole, and its their fault if crummy product ships, then what you have is a QI team. Does the shit hit the fan every time software ships, and it never gets any better? You’ve probably confused QA with QI.
Much of this is good, but I find the revised nr 6 so vague as to be meaningless. And I strongly disagree with the change to nr 8, not least because it effectively denies a point to any team with home or remote workers by shifting the emphasis to office design.
Also, Joel was (is?) one of the biggest opponents of open offices and all the false ideas about “fostering collaboration” that it is usually justified with. The fact that the author still clings to that notion pretty much disqualifies the whole test for me.
I’ve predominately worked in open plan offices, and definitely don’t think that they’re the answer to collaboration. From my experience, I’m more inclined to believe that they’re a short-cut silver bullet used by management to try and make collaboration happen, as I allude to in the article.
Hey. I’m the author of the article. Firstly thanks for taking the time to read the article, and to provide feedback.
I agree with you on the wording on #6. It’s been a couple of months since I wrote this, and looking back at the article now it’s really vague, both in title and the way I describe it. I have trouble with schedules, probably because I’ve been burnt in the past through working with managers and product owners (agile, yup) that are more interested in sticking to a schedule rather than understanding whether they’re still building and delivering the right thing for the users, customers or business. I was trying to convey the need to be able to respond to change, and to ensure that as an engineering team you have a consistent way of measuring value, and delivering the things that are most valuable.
In my mind #8 doesn’t exclude remote workers. Environment is used in a very broadest sense of the word. As I explain in the article, it’s more than just co-locating teams (although I think this can help). It’s about the wider corporate environment that the engineering group sits in - are decisions communicated well, understood and well reasoned? are opinions sought? can people speak their mind? how do people solicit and give feedback? how are people incentivised? That’s the kind of thing I was going for, and probably didn’t describe it well enough.
Author here. Thanks for feedback and discussion around this piece.
A little context. I wrote this a couple of months ago as I was in the process of leaving my previous company. I was running down my notice period, and I think some of the frustrations I was experiencing at the time are shown in the article. Maybe sitting on it a while might have been a better idea?
Anyways. My aim in writing this was to funnel some of my ideas around what makes good software engineering teams. I come from a background of working on consumer facing web products, usually in environments using agile techniques and practices. I think that this is reflected in the list, and I’m really happy to hear about some differences of opinions.
There are a lot of great open source software tools, but that blanket isn’t enough to cover you. Having the best tools money can buy includes support contracts (possibly to upstream devs), service management (datacenter or various IaaS/PaaS services) and excellent dev hardware (laptop?)
This is more what I was thinking about when I wrote the article. I think that open source tools, software definitely has it’s place in engineering, and should probably be preferred in a lot of cases.
I’ve always been attracted to the idea of 80/20 time, and I know that (especially at Google) the implementation never lives up to the idea.
Still, I think that the intent is noble: give the developer some ordained “free time” to get their creativity going again. In that spirit, I think that experimenting with different development tools (IDEs or editors, continuous integration tools, etc) should be encouraged and discussed. After a milestone, as the team has a retrospective on what worked and what didn’t, I think tools should be a part of the discussion. Did Sarah discover that Jenkins is more useful than Bamboo? Hey, look, Chuck learned about oh-my-zsh, you know that sort of thing.
Several chuckles from me, but maybe the realization is software engineering is becoming less and less monolithic is important here.
Things like 6, 7, 8, and 10 being changed from Joel’s test are pretty huge departures from my work environment, and I have strong rebuttals for each one… but that’s only in the context of my work. Curious who can defend things like 8.
diff:
I agree with the changes to (2) and (3).
The change to (7) seems to misunderstand what Joel originally meant by spec, which is a more thorough formal communication than at least the vast majority of automated specs or “BDD” implementations I’ve seen in the wild.
The changes to (6), (8) and (10) seem to be direct disagreements with Joel.
The change to 10 is incorrect. The best way to make sure that nobody is responsible for testing is to make everyone responsible for testing.
Few people I think have worked with a proper QA/testing group…if they had, they would not dismiss the utility of same so easily.
I see your point here, and I think I could have worded it better.
I’ve worked with (what I believe to be) an excellent testing group, however I stand by my claim that testing should be part of everyone’s role. That’s more of what I was thinking, and maybe that’s how I should have worded that one.
While I agree, I do think that having programmers write tests for their code is essential. Our current arrangement:
QA is invaluable in my opinion, especially for large systems with a good deal of ins and outs. In my experience, the developer (i.e. me) is not always able to see the edge cases. At the same time, having regressions and specs and programmers writing their own tests also ensures that QA isn’t finding the same bug repeatedly.
A good QA group can definitely be beneficial, but they need to be supplemental not the main test effort. It’s easy to make everyone responsible for testing with policy. Landing new code without a functional test? Fixing a bug without a regression test? Your patch doesn’t get past review.
Isn’t this is a No True Scotsman argument?
[Comment removed by author]
What are their goals and how is their performance judged?
Quality Assurance is a reporting organization.
Their job is to provide evidence that the quality of the product is understood, and to predict how the product will perform when it is deployed, so that the business can make an informed decision on when to release product. Their performance is measured by the strength of that evidence and the accuracy of that prediction.
As such, a QA team is responsible for checking that software matches its requirements, and that those requirements make sense. The QA team provides oversight for both the development and product management teams, and is the third side of the DEV-PM-QA product development structure.
QA is NOT the same thing as “making certain the product works”- that would be Quality Ensurance. If what you have is a group of engineers who are measured by defects found over time, then you have a QE team. You might still ship software, but your developers and testers are likely at odds with each other all the time. Do QA and Dev hate each other? You’ve probably confused QA with QE.
QA is certainly NOT there to provide a group of scapegoats to take the blame when bugs happen in the field- that would be Quality Insurance. If you think your QA / test people are somehow lower on the ‘old totem pole, and its their fault if crummy product ships, then what you have is a QI team. Does the shit hit the fan every time software ships, and it never gets any better? You’ve probably confused QA with QI.
Developers are often judged by features/defects, or often enough just features/time disregarding defects entirely. QA is judged by defects/time.
There’s responsible for, and there’s tasked with.
I agree that there should be dedicated testing resources, otherwise it gets orphaned, but everyone IS jointly responsible for testing the product IMO.
Much of this is good, but I find the revised nr 6 so vague as to be meaningless. And I strongly disagree with the change to nr 8, not least because it effectively denies a point to any team with home or remote workers by shifting the emphasis to office design.
Also, Joel was (is?) one of the biggest opponents of open offices and all the false ideas about “fostering collaboration” that it is usually justified with. The fact that the author still clings to that notion pretty much disqualifies the whole test for me.
I’ve predominately worked in open plan offices, and definitely don’t think that they’re the answer to collaboration. From my experience, I’m more inclined to believe that they’re a short-cut silver bullet used by management to try and make collaboration happen, as I allude to in the article.
Hey. I’m the author of the article. Firstly thanks for taking the time to read the article, and to provide feedback.
I agree with you on the wording on #6. It’s been a couple of months since I wrote this, and looking back at the article now it’s really vague, both in title and the way I describe it. I have trouble with schedules, probably because I’ve been burnt in the past through working with managers and product owners (agile, yup) that are more interested in sticking to a schedule rather than understanding whether they’re still building and delivering the right thing for the users, customers or business. I was trying to convey the need to be able to respond to change, and to ensure that as an engineering team you have a consistent way of measuring value, and delivering the things that are most valuable.
In my mind #8 doesn’t exclude remote workers. Environment is used in a very broadest sense of the word. As I explain in the article, it’s more than just co-locating teams (although I think this can help). It’s about the wider corporate environment that the engineering group sits in - are decisions communicated well, understood and well reasoned? are opinions sought? can people speak their mind? how do people solicit and give feedback? how are people incentivised? That’s the kind of thing I was going for, and probably didn’t describe it well enough.
One possible way to salvage the intent of #6 while making it more concrete would be to phrase it in terms of cost of delay.
Author here. Thanks for feedback and discussion around this piece.
A little context. I wrote this a couple of months ago as I was in the process of leaving my previous company. I was running down my notice period, and I think some of the frustrations I was experiencing at the time are shown in the article. Maybe sitting on it a while might have been a better idea?
Anyways. My aim in writing this was to funnel some of my ideas around what makes good software engineering teams. I come from a background of working on consumer facing web products, usually in environments using agile techniques and practices. I think that this is reflected in the list, and I’m really happy to hear about some differences of opinions.
No, no, no. We have the best tools money can’t buy.
So many projects I have seen derailed by….
How about instead, “Pay’s upstream devs / contributes to the best tools money can’t buy.”
There are a lot of great open source software tools, but that blanket isn’t enough to cover you. Having the best tools money can buy includes support contracts (possibly to upstream devs), service management (datacenter or various IaaS/PaaS services) and excellent dev hardware (laptop?)
This is more what I was thinking about when I wrote the article. I think that open source tools, software definitely has it’s place in engineering, and should probably be preferred in a lot of cases.
I’ve always been attracted to the idea of 80/20 time, and I know that (especially at Google) the implementation never lives up to the idea.
Still, I think that the intent is noble: give the developer some ordained “free time” to get their creativity going again. In that spirit, I think that experimenting with different development tools (IDEs or editors, continuous integration tools, etc) should be encouraged and discussed. After a milestone, as the team has a retrospective on what worked and what didn’t, I think tools should be a part of the discussion. Did Sarah discover that Jenkins is more useful than Bamboo? Hey, look, Chuck learned about oh-my-zsh, you know that sort of thing.
Several chuckles from me, but maybe the realization is software engineering is becoming less and less monolithic is important here.
Things like 6, 7, 8, and 10 being changed from Joel’s test are pretty huge departures from my work environment, and I have strong rebuttals for each one… but that’s only in the context of my work. Curious who can defend things like 8.
[Comment removed by author]