1. 9

  2. 3

    I used Datomic to build Cook, a multitenant preemptive batch & Spark scheduler for Mesos. We’ve used all kinds of features of Datomic, such as the object-like API, many different query features, raw index access, transaction functions, and log tailing. We also developed our own core.async API for Datomic to simplify its usage from a concurrent application (especially with retries).

    I’d be happy to answer any questions about the experience.

    1. [Comment from banned user removed]

      1. 1

        We open-sourced some code to do that. For example, to have retrying core.async-API transactions for Datomic, “update-in” like functionality, and idempotency, this whole namespace has that functionality: https://github.com/twosigma/Cook/blob/master/scheduler/src/metatransaction/utils.clj

      2. 1

        Nice! I don’t have too much experience with Datomic, so I just have two pretty general questions:

        • Are you using the free version of Datomic in production? If so, have you had any issues with the embedded storage engine vs. the external ones like SQL or DynamoDB?
        • Have you open-sourced your core.async API for Datomic as an independent project? I remember trying to find one back in the day when I was exploring Datomic, and I was quite baffled not to find any leads.
        1. 2

          We’re using Datomic Pro with Riak–it’s worked quite well for us for the past ~18 months.

          Our API is here: https://github.com/twosigma/Cook/blob/master/scheduler/src/metatransaction/utils.clj We also include some schema snippets if you want to have all-or-nothing commits across many transactions (for giant non-atomic but isolated transactions).

          transact-with-retries and update are the 2 most powerful functions. They have several variants for blocking and core.async compatible forms. Also, there’s a helper for making idempotent transactions. We built all this to integrate with core.async and enable our application to have zero issues during Datomic failovers, if a transaction were to fail.