I used Datomic to build Cook, a multitenant preemptive batch & Spark scheduler for Mesos. We’ve used all kinds of features of Datomic, such as the object-like API, many different query features, raw index access, transaction functions, and log tailing. We also developed our own core.async API for Datomic to simplify its usage from a concurrent application (especially with retries).
I’d be happy to answer any questions about the experience.
Nice! I don’t have too much experience with Datomic, so I just have two pretty general questions:
Are you using the free version of Datomic in production? If so, have you had any issues with the embedded storage engine vs. the external ones like SQL or DynamoDB?
Have you open-sourced your core.async API for Datomic as an independent project? I remember trying to find one back in the day when I was exploring Datomic, and I was quite baffled not to find any leads.
transact-with-retries and update are the 2 most powerful functions. They have several variants for blocking and core.async compatible forms. Also, there’s a helper for making idempotent transactions. We built all this to integrate with core.async and enable our application to have zero issues during Datomic failovers, if a transaction were to fail.
I used Datomic to build Cook, a multitenant preemptive batch & Spark scheduler for Mesos. We’ve used all kinds of features of Datomic, such as the object-like API, many different query features, raw index access, transaction functions, and log tailing. We also developed our own core.async API for Datomic to simplify its usage from a concurrent application (especially with retries).
I’d be happy to answer any questions about the experience.
[Comment from banned user removed]
We open-sourced some code to do that. For example, to have retrying core.async-API transactions for Datomic, “update-in” like functionality, and idempotency, this whole namespace has that functionality: https://github.com/twosigma/Cook/blob/master/scheduler/src/metatransaction/utils.clj
Nice! I don’t have too much experience with Datomic, so I just have two pretty general questions:
We’re using Datomic Pro with Riak–it’s worked quite well for us for the past ~18 months.
Our API is here: https://github.com/twosigma/Cook/blob/master/scheduler/src/metatransaction/utils.clj We also include some schema snippets if you want to have all-or-nothing commits across many transactions (for giant non-atomic but isolated transactions).
transact-with-retries and update are the 2 most powerful functions. They have several variants for blocking and core.async compatible forms. Also, there’s a helper for making idempotent transactions. We built all this to integrate with core.async and enable our application to have zero issues during Datomic failovers, if a transaction were to fail.