1. 13
  1.  

  2. 2

    This is the kind of distributed database I’d want to use. Dealing with data loss from extremely fringe eventual consistency bugs is not my definition of fun.

    Also, the keys having a total order is really nice! :D

    1. 3

      I don’t see FoundationDB hitting the same use cases as an eventually consistent DB. I’m not actually sure what the use case is for FoundationDB. The database does not support multiple data centers and the company actively tries to talk people out of expanding across datacenters[1]. Things that would require the kind of write performance they are offering are going to be pretty big, I would imagine. You wouldn’t host Twitter in 1 datacenter, for example (to take a comparison the author makes).

      I think it’s cool that they are achieving this kind of performance. I just don’t understand want problems it’s solving. I would rather have eventual consistency where I can setup clusters around the world and sync them, giving me horizontal scalability of operations, than be stuck in 1 database cluster in one datacenter and have to get as much performance out of that as possible.

      [1] http://highscalability.com/blog/2014/11/12/three-reasons-you-probably-dont-need-multi-data-center-capab.html

      1. 1

        I’m not actually sure what the use case is for FoundationDB.

        It goes fast, and has multi entity transactions. Already with transactions it’s better than Amazon DynamoDB, and there are lots of use cases for that. DynamoDB also doesn’t have multi datacenter replication.

        Not every application needs to be multi datacenter. Tons of applications have exactly zero need for that. Not a lot of mega-scale web 42.0 apps, but those certainly aren’t the only applications that use a lot of data. Your use case is always on ecommerce (if I recall correctly), and you didn’t see a use case for Redis Cluster either, which is hardly surprising given your domain.

        I would rather have eventual consistency where I can setup clusters around the world and sync them, giving me horizontal scalability of operations, than be stuck in 1 database cluster in one datacenter and have to get as much performance out of that as possible.

        If these benchmarks are accurate, you don’t need multi datacenter capability for horizontal scalability. Use case wise, data exploration and analysis would be extra flashy with a database with this kind of latency and throughput. Real time analysis would be way more real time.

        All that besides, the transaction coordination system is still quite interesting.

        1. 5

          Not every application needs to be multi datacenter. Tons of applications have exactly zero need for that.

          I agree. But how many of those apps need to do 14 million writes per second? It’s unclear to me who needs this kind of performance in one datacenter that doesn’t also need to scale horizontally, at the very least for availability.

          If these benchmarks are accurate

          I do question the results of the benchmark too. The test was only run for 5 minutes. That is far too little time for a database test. On top of that, this was in very ideal circumstances, the data fitting easily in RAM. I’d like to see a run for around a week, with data larger than can fit in RAM. Not that I think the results are a lie, more than I’m not sure it is covering a realistic workload.

          1. 2

            More distant clients would probably be good too. They’re comparing to twitter, for example, but twitter clients aren’t all running in the same cluster. Running the DB in one AWS region and the load generator in a separate would be a good test (like say US East vs US West).

            1. 1

              I don’t think that many need 14 million writes per second, I think it’s more than you can have less hardware and still get really good performance, with the option to scale up fairly easily in the future. You certainly don’t need to employ hundreds of servers in a base case deployment of the database, the benchmarks are more about saying that you could if you wanted to for whatever reason.

              Also, RAM is the new disk. ;)

            2. 3

              I agree with you in general, but it’s worth considering that only being in one datacenter is a resiliency hazard. If your datacenter gets smashed by a hurricane for example, you’re going to see nontrivial downtime, even if you can handle 14M writes per second.

              With that said, I don’t think that foundation DB is anti-multi-DC. I think they already have some partial support of it. From their docs:

              Multiple-datacenter support in this release is immature. If you are considering using multiple datacenters in production, we recommend that you contact us so we can work together for best results.

              1. 2

                With that said, I don’t think that foundation DB is anti-multi-DC. I think they already have some partial support of it. From their docs:

                I’ll believe it when I see it. The goal of the database seems at conflict with supporting multiple datacenters. And the article from FoundationDB on High Scalability seems to be trying hard to talk people out of multiple datacenters. I’ll be interested to see what they do though, clearly very capable people.

              2. 2

                DynamoDB also doesn’t have multi datacenter replication.

                DynamoDB only has multi-datacenter (well “facility”) replication. You can’t disable it. It’s not cross-region though. From documentation:

                The service replicates data across three facilities in an AWS Region to provide fault tolerance in the event of a server failure or Availability Zone outage.

                1. 4

                  I’m aware, I used to work on DynamoDB before I left Amazon, I was just simplifying. The big thing people want is actually multi region replication. Having 3 data centers right next to each other makes them susceptible to the same problems. Region wide failures aren’t unheard of on AWS.

                  1. 1

                    Thanks for clarification!