I was hoping for a real discussion of using Elixir at significant scale, alas. This is an intro article from an agency that, while employing a number of Elixir heavy hitters, mostly deals in agency-scale projects.
I was hoping for the same. I could not really find content about how to scale up a gen_server. If somebody has any good resources it would be great to see it here.
There’s no “scaling up” a genserver, a genserver is one process just like any other one on the VM. To scale past a bottleneck where one genserver can’t process its messages fast enough (a common Elixir/Erlang scaling problem), you need to figure out how to use more than one genserver to do the work. That means you can’t call the genservers by name anymore, but the process Registry is good for that
I’ve been using a pattern lately where, in the genserver’s init(args) function, I register the process under a name that is passed in the args parameter. Then I launch the genserver under a supervisor. If the process dies, its entry will be removed from the registry, and when the supervisor launches another genserver to replace it, that server will register itself automatically. I like it so far.
If your gen_server does not need a lot of state to start it up (or side steppable with persistent_term), it may be easier to use simple_one_for_one and spawn a new process for each unit of work and have the source of the request monitor the process to handle retries. This works especially well when the inputs are binaries as then a copy of that data is not produced so starting up becomes cheaper and GC is now just the process destruction.
Routing is difficult and sometimes it is best just to skip it altogether, especially when spawning is really really cheap.
timer:tc(fun() -> Self = self(), Pid = spawn(fun() -> Self ! ok end), receive ok -> ok end end).
I’m not aware of any rules like “you must write elixir/Erlang exactly this way to scale horizontally”. It scales automatically, part of BEAM. Just write code in these languages and don’t try to go outside the box (NIFs, for example) and you’ll be fine is my understanding.
this works, although as with any sharded thing, it’s helpful if you design to permit sharded stuff to stay in-shard as much as possible. I mention this because erlang has “free” distributed systems built in, so reaching across shards is trivial, even accidentally. There’s an old anecdote of a company that moved a bunch of servers from London to another continent, booted them up, and the app was mysteriously slow. Come to find out, they had accidentally left the database server behind, and erlang was seamlessly accessing it across the internet with appropriate speed penalty. Made for confusing debugging.
I was hoping for a real discussion of using Elixir at significant scale, alas. This is an intro article from an agency that, while employing a number of Elixir heavy hitters, mostly deals in agency-scale projects.
I was hoping for the same. I could not really find content about how to scale up a gen_server. If somebody has any good resources it would be great to see it here.
There’s no “scaling up” a genserver, a genserver is one process just like any other one on the VM. To scale past a bottleneck where one genserver can’t process its messages fast enough (a common Elixir/Erlang scaling problem), you need to figure out how to use more than one genserver to do the work. That means you can’t call the genservers by name anymore, but the process Registry is good for that
I’ve been using a pattern lately where, in the genserver’s
init(args)function, I register the process under a name that is passed in theargsparameter. Then I launch the genserver under a supervisor. If the process dies, its entry will be removed from the registry, and when the supervisor launches another genserver to replace it, that server will register itself automatically. I like it so far.You’ve just posted a very short introduction to the concept of how to scale up the use of gen_server(s, plural) 😸
If your
gen_serverdoes not need a lot of state to start it up (or side steppable withpersistent_term), it may be easier to usesimple_one_for_oneand spawn a new process for each unit of work and have the source of the request monitor the process to handle retries. This works especially well when the inputs are binaries as then a copy of that data is not produced so starting up becomes cheaper and GC is now just the process destruction.Routing is difficult and sometimes it is best just to skip it altogether, especially when spawning is really really cheap.
So what are the best patterns for scaling horizontally with Erlang/Elixir?
I’m not aware of any rules like “you must write elixir/Erlang exactly this way to scale horizontally”. It scales automatically, part of BEAM. Just write code in these languages and don’t try to go outside the box (NIFs, for example) and you’ll be fine is my understanding.
My impulse is to load balance the way I would any python code.
this works, although as with any sharded thing, it’s helpful if you design to permit sharded stuff to stay in-shard as much as possible. I mention this because erlang has “free” distributed systems built in, so reaching across shards is trivial, even accidentally. There’s an old anecdote of a company that moved a bunch of servers from London to another continent, booted them up, and the app was mysteriously slow. Come to find out, they had accidentally left the database server behind, and erlang was seamlessly accessing it across the internet with appropriate speed penalty. Made for confusing debugging.
That’s a nice story. Would make a good post of it’s own.
What do you mean? Run multiple python processes and use haproxy or something?
edit: this is probably what you’d want to know https://www.poeticoding.com/the-primitives-of-elixir-concurrency-full-example/
Yea basically.