1. 21

3 part series, a mistake many of us made or inherited. :(

And if you like schadenfreude, watch Hackernewses repenting here.


  2. 15

    A company I worked at a few years back migrated an internal service to MongoDB. It took a 9 node MongoDB cluster to replace an single, old, untuned instance of Postgres where Postgres was the “bottleneck”. Besides it being a slog of a migration, the only thing outstanding that I recall is that, when I asked about the performance difference between the “old” solution and the MongoDB solution, the developers all said that it’s unfair to compare the two solutions because they’re so different.

    I asked them flat out how long the task took before and how long it took after the migration - it went from 5 hours on Postgres to 7.5 hours on the MongoDB cluster.

    I was there for only a few months after that migration, but while I was there they never did get that run time down to where Postgres was.

    1. 11

      Maybe I’m floating away, but there seems to be a strong similarity between MongoDB’s “schemaless” design and dynamic typing: they’re great for experiments and rapid development but put off rigorous thinking about the problem domain at the expense of maintainability.

      1. 4

        There is an interesting blog post about voluntarily crippling your prototype to prevent it from making it to production. You benefit from rapid prototyping tools (Mongo, Python/Ruby) until the time to build the “real thing”‘comes and you get to learn from everything you’ve built so far with no risk to pushing fragile technologies to production. I like this idea very much.

        1. 21

          For an MVP I once titled a database “PrototypeDBDoNotUseForProduction”

          It’s in production :(

          1. 2

            If a programming language happens to be suited for getting results fast, that doesn’t mean it’s only a “rapid prototyping tool”.

            Just write the real thing right from the start, but be disciplined enough to establish a solid-enough foundation. Iterate as necessary.

            1. 1

              Maybe one of the issues is that you often don’t know what the right thing is. Malleable technologies that can do “anything quickly” at the expense of disciplined thinking seem to be well suited for prototyping. They excel in the early days of the project and many of them slow you down once the project has grown significantly.

              1. 2

                Malleable technologies that can do “anything quickly” at the expense of disciplined thinking seem to be well suited for prototyping.

                You insist? :p

                Python doesn’t force you to write shitty code.

                Neither does Clojure, even though it’s good for at least certain kinds of prototyping and “exploratory programming”. Clojure also happens to be one of the very best programming languages out there.

                But if you protect yourself from yourself by intentionally crippling your prototype, you’ll lose out on everything you’d have written while writing your prototype in your production-language that you could reuse for the real thing. That’s silly too.

        2. 7

          Looking through the articles he linked, one thing I note is how few of them provide hard data to back up their claims. The only one that did was Etsy’s 3-parter (which was actually a part-anna-halfer), where they showed some benchmarks for how MongoDB performed under load. They didn’t, though, provide similar benchmarks for MySQL. Without that, you can’t argue that Mongo is more scalable.

          Come to think of it, a lot of engineering-shaped marketing material is the same way: arguing from first-principles or the first half of an experiment than rigorous treatments. I wonder if that makes them less effective. Given how successful Mongo was, probably not.

          1. 9

            Because many (most? all?) startups in the field aren’t businesses that have requirements, but rather supervised work therapy programs for undereducated overprivileged idiots. Think about the way banking used to be – a soft landing place for the dimwitted rich.

            1. 3

              I recall reading some thought leader saying that it’s okay if the business unit has so little vision it cannot articulate requirements to engineering, because “startups” or some hand-wavey nonsense.

              That’s fine and all, but if it’s the case…what do they do?

            2. 4

              As Richard Hipp of Sqlite puts it… NoSQL databases are post modern databases. They only offer opinions, not facts.

              I tend to the CJ Date view of the world…

              …if you constrain SQL even further, you leave the realm of data and queries and enter the realm of facts and logical inferences from those facts.

              1. 4

                For the same reason they once chose MySQL: it’s very easy to get started.

                1. 2

                  https://www.nemil.com/mongo/2.html :

                  I’ll challenge our community: design a Hacker News algorithm — or more likely, human assisted system — that favors content and comments that makes a better engineer and is less permeable to attack. Our current algorithm is an alpha on the way to a more mature algorithm that understands what engineers need to learn, not just what some want to publicize.

                  This makes the wrong assumptions that people participate in public forums in order to provide bits and pieces of a formal education that should be curated by moderators in order to better educate “engineers”.

                  1. 7

                    Excellent quote.

                    Putting a new site in place won’t fix this, however. We should disparage the practices that gave rise to it, such as the fetishization of technology (“dude, what’s your stack?!”), the infantile hype cycle, pernicious anti-intellectualism in all it’s forms, immature engineering practices, and public acknowledgement that experience trumps the flavor-of-the-week framework. Talking about on Internet forums isn’t worth much at all.

                    In short, we need to grow up as a profession.

                  2. 2

                    This quote from part 2 resonates with me:

                    Schemaless does not mean no schema; instead, it means an implicit schema in the app (a particularly challenging misnomer for anyone outside our industry)

                    I’ve tried to make that argument many, many times in my career. My preferred phrasing: “If you can query it meaningfully, it’s structured/schema’d data, regardless of how you store it”.

                    I continue to be astonished with how frequently that perspective is considered novel, and how often developers try to argue against it.

                    1. 4

                      I say something along the lines of, “It has a schema, the database just doesn’t know what it is.” Or in particularly bad instances, “It has a schema, you just don’t know what it is.”

                    2. -2

                      Insert relevant JWZ quote here?