No, and this also isn’t important in Rama’s model since PStates are not your source of truth. Depots are the source of truth and the definition of the data can and should be completely normalized. Any consistency issue in your PStates can be fixed with a recompute from depot data.
In RDBMS’s, referential integrity only protects against a subset of possible data integrity issues. When you need to denormalize your schemas in an RDBMS (due to being forced to for performance reasons), maintaining data integrity becomes an application problem. And this should never be the case for a supposed source of truth.
I don’t see how normalisation or denormalization affects whether I want transactions or not.
Can a user write into the depot with a transaction? Concretely: can they observe some state in the depot or a pstate at time T1 and then write something to the depot at time T2 only if that observed state is still valid?
Or do you get around it by having the depot be an atomic message log that you write higher-level messages to, and then each pstate can get access to that log?
Rama has two kinds of topologies, streaming and microbatching.
Microbatch topologies are cross-partition transactions for every PState in every case. Our bank transfer example relies on this.
Stream topologies are transactional for all PState changes on one partition of a module done in the same event, no matter how many.
You don’t need transactions for depots, since all data that must be atomic should just be in the same depot record. Like for bank transfer, you append one depot record that says “Transfer $10 from Alice to Bob” and not two depot records saying “Deduct $10 from Alice” and “Give $10 to Bob”.
The ACID page in the docs goes into this in more detail.
Rama is a platform for developing backends, general enough to handle all the computation and storage for pretty much any application at any scale. Rama is deployed as a cluster, and on that cluster you can deploy any number of Rama applications called “modules”. Module are programmed in either Java or Clojure.
Modules are event-sourced, so all information comes in through distributed logs called “depots”. In your module you code “topologies” which react to incoming data on depots to materialize any number of indexed datastores called PStates (“partitioned state”). PStates are defined as any combination of data structures and can thus represent any existing data model of any database plus infinite more.
All state in Rama is incrementally replicated, so if any nodes are lost a follower is ready to take over for that node immediately.
You can see short, heavily commented examples of using Rama for a variety of use cases in rama-demo-gallery.
Well, Rama’s not a VM. You express distributed code with Rama with its dataflow API, which is much more concise and expressive than Erlang-style message passing. It also provides higher-level guarantees about data processing than Erlang (e.g. microbatch topologies in Rama have exactly-once semantics and are cross-partition transactions for completely arbitrary distributed code in every case).
I thought it was something like SpacetimeDB - a system for building multiplayer applications (games or otherwise, though Spacetime seems to be for games) where your logic lives “inside” the datastore rather than being a separate app that reads/writes from it.
SpacetimeDB describes itself like this (if Rama’s different this won’t be helpful):
You can think of SpacetimeDB as both a relational database and a server combined into one. Instead of deploying a web or game server that sits in between your clients and your database, clients connect directly to the database and execute your logic inside the database itself.
It’s unclear to me if there’s referential integrity across these various PStates. Or transactions across them. Anyone know?
No, and this also isn’t important in Rama’s model since PStates are not your source of truth. Depots are the source of truth and the definition of the data can and should be completely normalized. Any consistency issue in your PStates can be fixed with a recompute from depot data.
In RDBMS’s, referential integrity only protects against a subset of possible data integrity issues. When you need to denormalize your schemas in an RDBMS (due to being forced to for performance reasons), maintaining data integrity becomes an application problem. And this should never be the case for a supposed source of truth.
I don’t see how normalisation or denormalization affects whether I want transactions or not.
Can a user write into the depot with a transaction? Concretely: can they observe some state in the depot or a pstate at time T1 and then write something to the depot at time T2 only if that observed state is still valid?
Or do you get around it by having the depot be an atomic message log that you write higher-level messages to, and then each pstate can get access to that log?
Or something more exotic?
Rama has two kinds of topologies, streaming and microbatching.
Microbatch topologies are cross-partition transactions for every PState in every case. Our bank transfer example relies on this.
Stream topologies are transactional for all PState changes on one partition of a module done in the same event, no matter how many.
You don’t need transactions for depots, since all data that must be atomic should just be in the same depot record. Like for bank transfer, you append one depot record that says “Transfer $10 from Alice to Bob” and not two depot records saying “Deduct $10 from Alice” and “Give $10 to Bob”.
The ACID page in the docs goes into this in more detail.
Can anyone explain what Rama actually is? I’ve been aware of it for two years, occasionally read around etc. and it just seems baffling.
Rama is a platform for developing backends, general enough to handle all the computation and storage for pretty much any application at any scale. Rama is deployed as a cluster, and on that cluster you can deploy any number of Rama applications called “modules”. Module are programmed in either Java or Clojure.
Modules are event-sourced, so all information comes in through distributed logs called “depots”. In your module you code “topologies” which react to incoming data on depots to materialize any number of indexed datastores called PStates (“partitioned state”). PStates are defined as any combination of data structures and can thus represent any existing data model of any database plus infinite more.
All state in Rama is incrementally replicated, so if any nodes are lost a follower is ready to take over for that node immediately.
You can see short, heavily commented examples of using Rama for a variety of use cases in rama-demo-gallery.
Does this mean it’s like BEAM for the JVM, letting you write distributed Clojure like Elixir?
Well, Rama’s not a VM. You express distributed code with Rama with its dataflow API, which is much more concise and expressive than Erlang-style message passing. It also provides higher-level guarantees about data processing than Erlang (e.g. microbatch topologies in Rama have exactly-once semantics and are cross-partition transactions for completely arbitrary distributed code in every case).
I thought it was something like SpacetimeDB - a system for building multiplayer applications (games or otherwise, though Spacetime seems to be for games) where your logic lives “inside” the datastore rather than being a separate app that reads/writes from it.
SpacetimeDB describes itself like this (if Rama’s different this won’t be helpful):