As the poor sod who was instructed to make this happen and made it happen “One Source Tree, Many Products”, I can comment with a lot of experience.
One Source Tree, Many Products. Many small software projects that survive to grow into large, monolithic projects are eventually broken up into components. It’s easy to do this by taking the existing source tree and building parts of it, and it’s also wrong. Builds that slice up a single source tree require too much discipline to maintain and too much mental effort to understand. Break your build into separate projects that are built separately, and have each build produce one product.
The core sentence is here.
Builds that slice up a single source tree require too much discipline to maintain and too much mental effort to understand.
Yup. It requires deep and sustained effort to untangle the spaghetti the ordinary programmer produces, and a way of enforcing the discipline of keeping those parts that are untangled from becoming tangled again.
So how did I do it?
I grabbed control of the build system. Given I had experience of writing a couple of other build systems, I could rapidly give shiny things to the team whenever they asked.
I enforced cleanliness. It’s a C project, so the first bit of cleanliness I enforced via the build system is that each header file (. h file) must compile by itself, and that there are no cycle’s in the #include graph. ie. No a1.h #includes a2.h, which #includes a3.h, …., which #includes ai.h, which #include’s a1.h.
That took me quite a bit of work to achieve and then lock in place via the build system.
I would argue that this made the build system somewhat harder to understand, but the body of the code (a far larger beast worked on by far more people), much much easier to understand.
Trying to assign file by file out of thousands of files, line by line out of millions of lines of code to different products is an insane task, especially as you must assign everything they depend on.
Thus I partitioned the files (in the mathematical sense of each file in one and only one partition), into “Units of Assignment to Deliverables”. (UAss for short.)
I then uncovered the dependency graph and made it happen such that the dependency graph between units was also acyclic. (In this case I defined unit A depends on unit B iff a file in A #include’s “b.h”, a file in B.)
Once an acyclic dependency graph was achieved, I again locked it in place via the build system. (See the pattern, clean something up, lock it in place via the build. It’s a people process level thing. It works.)
Again I would argue from bitter experience that the work and complexity in the build system more than repayed itself by massively reducing the complexity in the code.
How do I know this? Because I was the poor sod doing the untangling. Every damn time I untangled a particularly horrendous knot…. I uncovered and fixed bugs. Including a surprisingly large number of thread races.
As the famous quote says: There are only two ways of writing defect free code, one is so simple there are obviously no defects…. and the other so complex there are no obvious defects.
I bet you, from years and years of experience, show me a big gnarly architectural tangle full of big cyclic dependencies (no matter whether preprocessor, compile, link or run time level)…. I will find bugs for you. Lots of them.
Break your build into separate projects that are built separately, and have each build produce one product.
Sound advice… as long as don’t care about defects. Or test burden. Or features that fit across the entire product line.
But I will agree with the author on one thing.
It takes a huge amount of sustained discipline and mental effort.
Lessons Learnt:
Modern DVCS’s are lovely. Mercurial is soooo nice. Mercurial Evolve is ruddy wonderful.
Code cleanliness matters. Lock every gain in place via your build process.
Test Burden matters. Most of our pain arises from places where people carved things up line-by-line via #ifdef spaghetti rather than at a block of files at a time. The utility unifdef is your friend.
As the poor sod who was instructed to make this happen and made it happen “One Source Tree, Many Products”, I can comment with a lot of experience.
The core sentence is here.
Yup. It requires deep and sustained effort to untangle the spaghetti the ordinary programmer produces, and a way of enforcing the discipline of keeping those parts that are untangled from becoming tangled again.
So how did I do it?
That took me quite a bit of work to achieve and then lock in place via the build system.
I would argue that this made the build system somewhat harder to understand, but the body of the code (a far larger beast worked on by far more people), much much easier to understand.
Trying to assign file by file out of thousands of files, line by line out of millions of lines of code to different products is an insane task, especially as you must assign everything they depend on.
Thus I partitioned the files (in the mathematical sense of each file in one and only one partition), into “Units of Assignment to Deliverables”. (UAss for short.)
I then uncovered the dependency graph and made it happen such that the dependency graph between units was also acyclic. (In this case I defined unit A depends on unit B iff a file in A #include’s “b.h”, a file in B.)
Once an acyclic dependency graph was achieved, I again locked it in place via the build system. (See the pattern, clean something up, lock it in place via the build. It’s a people process level thing. It works.)
Again I would argue from bitter experience that the work and complexity in the build system more than repayed itself by massively reducing the complexity in the code.
How do I know this? Because I was the poor sod doing the untangling. Every damn time I untangled a particularly horrendous knot…. I uncovered and fixed bugs. Including a surprisingly large number of thread races.
As the famous quote says: There are only two ways of writing defect free code, one is so simple there are obviously no defects…. and the other so complex there are no obvious defects.
I bet you, from years and years of experience, show me a big gnarly architectural tangle full of big cyclic dependencies (no matter whether preprocessor, compile, link or run time level)…. I will find bugs for you. Lots of them.
Sound advice… as long as don’t care about defects. Or test burden. Or features that fit across the entire product line.
But I will agree with the author on one thing.
It takes a huge amount of sustained discipline and mental effort.
Lessons Learnt: