FWIW, I disagree with the premise of the article but wanted to hear people’s comments.
I just…don’t understand this article. No one argues that a single, monolithic server is both easier and more capable of advanced features. Distributed systems and horizontal scaling is hard, and you often lose some functionality.
But you will outgrow your single server if you have any sizable amount of data (and actually need to do something with it). Vertically scaling delays the inevitable, and won’t even delay it that much. If you think you will scale out horizontally eventually, paying the price for a correct infrastructure upfront will be vastly easier than refactoring when things start to go pear shaped.
The article is basically saying “Scaling is hard, so we’re gonna ignore it”
“Scaling is hard, so we’re gonna ignore it”
I’m fairly certain that’s the MongoDB way.
I agree with you, but I think I have an understanding of why the author is thinking in this way.
In many software teams, there’s a strong disposition to be risk-averse. I think the root of this argument is centered around that. In particular, it seems that the author of the OP has hit a local optima (or is heading in that direction), which you correctly identify. At some point, he will grow to the ceiling, and the only way up will be out.
The other thing you correctly identify, I think, is the idea of long-term risk calculation. This is a tough one to sell to a lot of developer’s, because the zeitgeist of the modern development culture is firmly rooted in what is essentially a greedy approach to software design. Essentially the heresy I promote is – sometimes you shouldn’t do the simplest thing that could possibly work, sometimes you should do something complicated.
Now, I’m not saying that simplicity should be shunned, rather, I’m saying that you shouldn’t be dogmatic about doing the simplest thing. Clearly, many performance problems can be solved by throwing money at the problem, but typically it’s not the most efficient method of doing things, and the watchword ought to be; I think; efficiency, not velocity.
The central argument to this heresy is this – Think of risk and complexity as two functions from an action to a real value which ‘measures’ the risk-of-execution or complexity-to-execute (respectively) of the action. Then I claim that Complexity correlates with Risk, but does not determine it. Further, the constant of that correlation is not all that close to the 1 we usually assume. As a corollary, wanton reduction of complexity often leads to accrual of technical debt – that is to say often ‘simplicity’ is better termed ‘deferred complexity’, and it’s that deferred complexity that is the real risk, because it’s hidden, and an apparently simple action can lead to very expensive changes later.
I think there’s a lot of emphasize in the language of our culture on the idea of rapidity, often hidden under the guise of ‘being agile’ – but I think that’s disingenuous to the ideal of the original manifesto – favoring working software, and intelligently responding to change doesn’t mean ‘just do the simplest thing always’, that’s not responding to change, that’s dogma.
IMO, there are already other solutions to MongoDB that fit a similar niche which do offer easier scaling options (I am a Riak fan). So these kind of apologetics (I don’t mean that in the accusatory way since your post was rather general and could apply to anyone talking about defending their local optima) don’t bear much weight when considering more of the database choices out there.
Following much criticism, MongoHQ pulled the original article on 2014/03/16 and rewrote it as the following: http://blog.mongohq.com/how-we-scale-mongodb/
That would explain the 404. It would have been nice of them to keep the original up and linked to the updated piece.
Here’s the Google cache of the original article:
Looks like it didn’t make it to archive.org.