-
One does not need ACID to solve the problem the author describes and ACID is likely overkill for the issue. There are other options such as causal consistency or RAMP transactions which are much light than ACID but still strong than eventual consistency.
But the author is spot on in that transaction boundaries are a big problem that groups either ignore or don’t address in their design.
Pet peeve though:
but I think you will find that they all have issues, and none are simpler or easier than an ACID database
and
ACID transactions are usually simpler than eventual consistency or distributed transactions
- Simpler in what regard? Production ACID transactions are very complicated to implement, they require a lot of machinery. They also put a lot of constraints on the data. And they can fail in surprising ways. EC is extremely simple but it pushes complexity to the user. So a blanket “X is simpler than Y” statement here doesn’t make much sense IMO.
- ACID transactions can be distributed transactions to the second quote is a bit confusing.
-
Thanks for the feedback. I have never heard of RAMP transactions before, so I’m definitely going to read up on that. Right now I don’t see how the problems can be solved without the A, C and I of ACID though.
I could soften the wording a bit. What I’m actually rallying against is pushing the complexity of transactions to the application code, instead of handling it in the database.
You’re right about #2. I’ll update the article.
(changes here: https://github.com/KMahoney/kmahoney.github.io/commit/605ae0c11a73fcf7e1ffe717d08d8c803a6dd3be)
-
Right now I don’t see how the problems can be solved without the A, C and I of ACID though.
That doesn’t necessarily mean you need the rest of ACID, though. And the rest of ACID is prohibitive. Separate services owning related data, for example, is very difficult if-not impossible. With causal consistency you can spread data out over multiple services and as long as they offer the data semantics needed (and you’re ok spending the bit of extra cost to implement it) it’s not a problem.
The key observation, IMO, for causal consistency or RAMP transactions is that the actual data in the DB doesn’t need to be consistent, you just need to be able to find a consistent version of it. So if you create/updated data in one place you need to either update it the other place or make sure you can get the state of data pre-update in both.
-
-
[Comment removed by author]
-
Do you mean as presented in the article? With services? Well, I don’t exactly know what you need to do. Entire books have been written on the subject. It’s a really hard problem.
You can look at how something like CockroachDB handles a similar problem.
-
I recall some platforms having a concept of a distributed transaction coordinator, which implies another piece of software to run, along with all your services implementing a protocol for rollback in case of failure.
At this point, you might as well just accept you are full-on Enterprisey. :)
-
-
You use an actual transaction - either distributed, or by having a single point of coordination. E.g. store both the user and the profile in a database using three-phase commit with an appropriate transaction isolation level. Or some fancy paxos/raft-based distributed transaction protocol.
Much better to structure your data such that the same record represents both the user and the profile, IMO.
-
-
I’m working on a WebGL RPG game, and I released the backend renderer: https://github.com/KMahoney/funscene
Still early days!
-
Absolutely wonderful article!
The only nit I will pick is that the idiomatic C code for something where we care about failure is something like:
error_t x(int *y, size_t n, int z, size_t* indexOut) { /* implementation elided */ }Thus, we separate the error from the data returned. I think the overall points they make still stand, though.
-
-
I won’t presume to know how widespread it is, but it’s done exactly to fix the cases where a sentinel value is still theoretically within the range of the data.
I dislike returning -1, for example, because if doing pointer arithmetic or something like that one can still mistakenly do math without the compiler complaining. An explicit error value and type being returned cannot be abused as readily, at least by accident.
I especially dislike sentinel values in the case of, say, dedicated routines for handling integer operations and catching overflows and underflows–those are cases where, for example, (-1) might conceivably show up during normal use.
In the case of finding indices of something, the index type should be
size_t, because (somehow) max int there could still be a valid index. It’s paranoid, but it is what it is.
-
-
-
I really like Nix, but I found the advantage with homebrew is that the packages are configured to integrate well with OS X. For example, I had problems with SSH keys when using the Nix version of git.
Maybe I’ll give it another go to see if it’s improved.
-
<rant>To be frank, as a backend engineer, the rotating door of JavaScript libraries is why it’s hard to take the community seriously. This is a complaint I see levied against backend engineers often and you frontenders do it to yourself. The choice of frameworks for backend things is large, sure, but mostly static. Python only has three or four serious choices and they have been around for awhile. Ruby has only a few as well. Etc. I have tried to learn frontend dev a few times and it’s just infuriating. By the time I’ve learned how to do what I want the in framework X.js the community has moved onto Y.js. It’s not even clear what the community is trying to optimize for! Is it speed? Download time? Simple APIs? I don’t know because the framework of the week is always so different from the previous. Next time you, a frontender, feel like backenders aren’t treating you like grown up professional devleopers, well damn straight, start acting like one, then.</rant>
-
My reaction to this article was, “Good fucking lord, another one?”
At this point, it’s nearly impossible to take frontend development seriously.
-
-
To be frank, as a backend engineer, the rotating door of JavaScript libraries is why it’s hard to take the community seriously. This is a complaint I see levied against backend engineers often and you frontenders do it to yourself.
Translation: I don’t understand why the “frontend community” (whatever that is) seems to do things differently than I would. Therefore, they aren’t worth taking seriously.
The choice of frameworks for backend things is large, sure, but mostly static. Python only has three or four serious choices and they have been around for awhile. Ruby has only a few as well. Etc.
There are only a few “serious choices” for frameworks in the frontend world as well. Angular, Ember, and React are the big three, and have been for a while. Most others are “flavor of the month” and never achieve critical mass. They just make a lot of noise.
I have tried to learn frontend dev a few times and it’s just infuriating. By the time I’ve learned how to do what I want the in framework X.js the community has moved onto Y.js.
What do you mean by “moved onto?” Blog posts are a poor metric for assessing where the “community” is actually at.
It’s not even clear what the community is trying to optimize for! Is it speed? Download time? Simple APIs? I don’t know because the framework of the week is always so different from the previous.
Is it really surprising that largest programming language “community” in the world is composed of lots of different people with lots of different priorities, levels of experience, knowledge of different paradigms? And that the frameworks reflect those differences?
The kind of JS framework that will make a Java developer happy is different from the kind that will make a Lisp developer happy, which is different again from what a Ruby developer wants. In JS, we have all of these communities bumping up against each other and doing things their own way, then cross-pollinating ideas.
It’s inevitable that there will be a lot of churn among smaller frameworks. And it’s inevitable that authors (and those ideologically similar to the authors) will want to promote their framework of choice. Thus the persistent “X is outdated, use Z” blog posts. This is further exacerbated by the fact that many frontenders come from a design and marketing background and are quite aware of the power of marketing and simply shouting louder.
Next time you, a frontender, feel like backenders aren’t treating you like grown up professional devleopers, well damn straight, start acting like one, then.
I can quite easily believe that there are more unprofessional JS developers than there are in other languages. (It has a pretty low barrier to entry, after all.) However, that doesn’t mean you get to just label JavaScript developers your inferiors and bask in your awesomeness.
-
Translation: I don’t understand why the “frontend community” (whatever that is) seems to do things differently than I would. Therefore, they aren’t worth taking seriously.
Actually, no. This is not my complaint at all. The rest of your response is all reasonable but this opener is completely missing the rant. It’s not that things are done differently than how I would do it, it’s that to an outsider the JS world is still solving problems via a Random Walk. I don’t like how the Java world solves a good bunch of problems, but there is method to the madness that I can at least understand.
-
Perhaps I phrased my response poorly, but your clarification here is actually what I understood you to mean. On rereading, I also see that my response comes off as hostile. Sorry about that.
I intended my opener to emphasize the idea that there is not a single JS community in the same way that there is a Ruby community, a Clojure community, etc. And the seeming randomness that results from this misidentification of the “JS community” as a coherent entity is more apparent than real.
-
-
-
I don’t think that’s fair. If you counted all the changes in every backend language, you’d see just as much chaos. It’s just that web frontenders only have one language to work with so every new tech gets put in the same bucket.
Plus, it looks like chaos, but over the long term you can see slowly converging best practices in the frontend world. Most new frameworks seem to agree that virtual-dom diffing and unidirectional data flow were a good idea for example.
-
I don’t think that’s fair. If you counted all the changes in every backend language, you’d see just as much chaos. It’s just that web frontenders only have one language to work with so every new tech gets put in the same bucket.
This comparison isn’t valid. Being in different language means you have no choice but to rewrite things. Many of the frameworks across different languages are copying each other and reusing ideas.
Your point is roughly that: the number of solutions to a problem that will be created is constant and it’s just a question of how concentrated it is (one language vs many languages). That might be observably true but doens’t seem like a good reason to do something.
Plus, it looks like chaos, but over the long term you can see slowly converging best practices in the frontend world.
How many iterations does it take to get MVC right? Maybe the actual stats disagree with me but I would expect the rate at which these new frameworks come out to be decreasing but that doesn’t seem to be the case.
-
How many iterations does it take to get MVC right?
Hard to tell, since there is not one universal “MVC” nor is there one universal “right way” to do it. Though we do seem to be moving away from “traditional” Rails-style MVC toward something more component-oriented.
I would expect the rate at which these new frameworks come out to be decreasing but that doesn’t seem to be the case.
My personal theory is that is because knowledge of front-end MV* is now dispersing more widely, leading more JS developers to try writing their own, just like jQuery clones were popular a couple of years ago.
-
-
-
You have three or four serious python choices NOW, but that wasn’t always the case.
Also remember that JS has so many more people using it because they’re basically forced to. The JS community is the largest by default, so it has the most variation in libraries of people wanting to code like they did in whatever their favorite/previous language was.
-
-
I’m working on a database written in Rust. I really like Rust. Sometimes the borrow checker stops you from doing something, and then you think about it and realise that’s it’s right - what you’re trying to do is unsafe. If i’d written it in C++ I would have made lots of mistakes.
http://kevinmahoney.co.uk/articles/
Mostly stuff about types and databases.