1. 23
  1.  

  2. 13

    With these kinds of texts, I always feel like someone took a “choose you own adventure”-book and just wrote down one of the more pessimistic storylines.

    A lot of these things can go right and a lot of these things can happen without a monorepos.

    1. 9

      Still not convinced that monoliths are a bad thing for most companies.

      1. 2

        The trade-offs between monolithic and non-monolithic approaches are well-known (monokernels vs. microkernels, linking vs. RPC/CORBA, SOA vs. whatever). There is a place for both and experience tells us what to use in what situation.

        Microservices are just a hype word for people to pimp up their resumés in the talk circuit, consultants to make money helping customers to ‘slay their monoliths’, companies to raise VC money for their microservice framework. Of course, the victims are companies that bought into the hype, would do fine with a monolith, and are now killed by microservice coordination issues.

        It’s a pattern that we see in the industry over and over again. The previous hype was to use map-reduce everywhere. As a result are a lot of folks using Hadoop clusters that are idle most of the day and when there is some work a reasonable implementation based on shared-memory parallelism would be much faster. But the ‘we use map-reduce/have big data’ box can be ticked off, so everybody is happy, right?

        1. 1

          Re: your hadoop example, I use sqlite on my laptop for data analysis. Set some tunables and you’re flyin’.

          I use these in my .sqliterc:

          PRAGMA cache_size = -1048576;
          PRAGMA journal_mode = MEMORY;
          PRAGMA page_size = 8192;
          PRAGMA synchronous = OFF;
          PRAGMA threads = 8;
          PRAGMA foreign_keys = ON;
          

          So, cache size 2mb -> 1gb, journal in RAM instead of disk, double the page size, disable synchronous writes, and use multithreaded algorithms. It stops sqlite from being crash safe, but when it’s just my personal laptop that doesn’t matter.

          I don’t think foreign keys help performance, they just make it harder to screw up your data. Bad data = bad results!

      2. 7

        Kind of confused by how much discussion there is over monoliths/microservices and repo organization. I’m not saying they’re unimportant considerations, but are they as important as the number of articles that exist on them?

        1. 5

          I posted a long response as a comment on the blog. The tl;dr is that this is a very naive use of a monorepo.

          1. 6

            Right, you can’t just check in all your code into a monorepo without any tooling or processes and expect things to turn out well. Just like how teams can’t each check their code into individual microrepos without any tooling or processes and expect things to turn out well.

            For some reason people always think version control is a silver bullet that can fix complexities of scale.

            1. 12

              You wouldn’t believe the kinds of things you get to see as a version control consultant. For instance: 50 branches, one per dev, kept open and unmerged for a year “because people kept breakng the build on trunk and got in each others way”. Then it was time for a release. Then they called us in. Not much to salvage…

              1. 2

                Oh dear.

                I had a boss who (the shop using SVN at the time) was amused at my eagerness to go branch off.

                “The branching is easy”, he’d said, “Tell me what you think when it’s time to merge.”

                git, for all of its faults, is pretty good at that.

                SVN is still nice for certain scenarios though that aren’t heavily branched.

                1. 2

                  It’s certainly possible with SVN, I’ve seen it done. Will for quality and discipline can go a long way.

                  1. 2

                    My recollection of my experience with svn merge was that the problem with it was just that it was arcane, buggy (e.g. IIRC if you try creating a branch in which there is a file called ‘foo’ and another in which there is a directory called ‘foo’ then misery happens) and super violent because it immediately does whatever idiot thing I asked for (*) on the shared repo instead of in some kind of safe sandbox like a local working copy or local clone of the repo.

                    (* assuming some non zero fraction of the time I get the wrong time range or something in the merge command on the first attempt)

                    I have had easy, painless branch merges with SVN. “Will for quality and discipline” had nothing to do with it; “using svn diff and patch to apply and edit patches manually instead” had everything to do with it.

                    git-svn, despite being a bit evil, was actually really useful because it let you try your merges locally and then examine the result before committing them.

                    1. 2

                      (e.g. IIRC if you try creating a branch in which there is a file called ‘foo’ and another in which there is a directory called ‘foo’ then misery happens)

                      Well, what do other systems do? Merging such nonsense is always a pain especially if it happens at scale of dozens of items, even in git/hg.

                      SVN might get better at this, eventually. At the moment, this is still being worked on. Subversion 1.10 will better handle cases like this where the node kind remains the same (file vs file, dir vs dir). Your scenario is still out of scope for that release, unfortunately – pay me good money and eventually i might fix that, too. But I have done more than enough work on this complex problem in my spare time while pretty much everyone else on the planet is just complaining from the peanut gallery.

                      1. 1

                        But I have done more than enough work on this complex problem in my spare time while pretty much everyone else on the planet is just complaining from the peanut gallery.

                        I don’t want to discourage or criticise you. I’m relating things that have happened to me personally, not calling your considerable abilities and hard work into question. My honest appraisal at this point is that:

                        • svn is a very high quality centralised VCS. It is enormously better than the competition that existed at the time it was created (though I’ve never had an opportunity to try p4 for comparison).
                        • I would still strongly recommend svn over any DVCS for doing version control of large binary files like film or game assets such as footage, meshes and textures.
                        • DVCS systems are, for source code, just a better model up until you hit scaling limits (which only mega-rich companies like Google, Facebook and Microsoft realistically ever do).
                        • svn now has way better merging than it used to. svn version 1.6 era merging was so bad that it caused an entire generation of developers to be scared of branching.

                        Well, what do other systems do? Merging such nonsense is always a pain especially if it happens at scale of dozens of items, even in git/hg.

                        They certainly don’t succeed at doing a merge when you do this; as you rightly point out there exists no sensible merge for this.

                        What all the DVCSes I’ve used did was hand me a working tree with both sides written into different files and then ask me to resolve the conflict.

                        IME svn used to do, upon attempting this merge, put my checkout into a completely broken state, in which none of the usual operations work any more, and which the available documentation didn’t explain very well how to get out of. For example, svn revert used to not work any more in this situation.

                        OTOH if I did the same with git or darcs, it:

                        • told me that the merge failed
                        • put the conflicting files and directories side by side in my working copy, keeping one with the name foo and giving the other a name like foo~HEAD or foo.~0~.
                        • asked me to resolve this by: deciding what I want to do, editing files to get the working tree into the shape I want to end up with, git adding and then git commiting to complete the merge. Note that git add and git commit are just the ordinary verbs for writing commits in git; this isn’t some kind of unusual weird situation where all the usual tools suddenly stopped working and I have to use completely different verbs from a whole different section of the manual to get out of it.
                        • if I really decided this was a terrible idea and want to back out, I could run git merge --abort to ditch this whole can of worms and put the repo back to the state it was in before attempting the impossible merge. This works reliably. In a rather atypical feat of decent UX, git even tells me about the possibility of doing this at the time the merge failed.
                        1. 1

                          For example, svn revert used to not work any more in this situation.

                          This must have been many many years ago. Such basic problems have long been fixed, mostly with the working copy redesign and rewrite that happened with Subversion 1.7 (which is old by now, first released in 2011).

                          I suppose many people saw some buggy behaviour from the SVN 1.5 days, then switched to something else, and still believe that what they have is actual working knowledge of SVN. It’s not, unless you have been using 1.8 or 1.9.

                          Granted, there are still many implementation bugs being found (here is a fun and recent one). But it’s not anywhere as horrible as it used to be. Thankfully :)

                          1. 1

                            Subversion 1.7

                            That didn’t exist when I started using subversion. I mentioned version 1.6 explicitly by name for this reason. ☺ I think, though I’m not sure, that I might even have started out with some version as early as 1.4.

                            I remember having to svn upgrade all the working copies when 1.7 came out! There were a few messes where behaviour differed, plus maybe a couple cases where the upgrade didn’t go smoothly for some reason.

                            I do not recall svn merge being noticeably more robust with 1.7 than it was with 1.6.

                            I think the company I was working for started moving to git in earnest around the time 1.8 was released.

                      2. 1

                        We’re talking organisation-wide level here, not about an assessment of how good svn merge is in detail, or for the individual. It’s not great, that’s known.

                        git-svn can be a tool for solving that, but that still needs adoption in your org.

                        1. 2

                          git-svn can be a tool for solving that, but that still needs adoption in your org.

                          I have no idea what you mean by this. One of git-svn’s historical advantages was that it could be used by a single person without anyone else knowing or needing to know that someone was using it.

                          Are we just talking about completely different things? Like you’re talking about the need for an organisation to avoid having long-running branches without at least merging trunk into them every time trunk is committed to, and I’m just talking about the much narrower problem that doing so with only svn merge is painful because svn merge is very imperfect?

            2. 3

              There is no such thing as a system without dependencies, there are only organisational tradeoffs as to how they are managed.

              1. 1

                Gosh. Exactly what happens in my company.

                We’ve had a big PHP monolith with a legacy framework, started to write Go services for the domains that were the less understood/slow/… so we could regain ownership. It worked so well that we did this to 60% of the monolith and since then, shipped many new with huge features. The iteration cycle is so quick that new features that were taking weeks are taking few days (sometime a day with the front/back/ops in one room to synchronise super-well on the new feature).

                The development created a mono-repo to be able to create PRs that would change several services at the same time, which tightens A LOT services together, as explained in the article, but at least it’s explicit (whereas when it was on several repos, it was hard to know, or maybe they shouldn’t…).

                Right now we’re having a very fast development cycle but we’re suffering on the operation side because services and teams need a lot of synchronization.

                When I read this article, and see what we’re doing, I really feel we’re missing something…