It’s missing an 11th gotcha, which I think is quite important for beginners:
Don’t make a GenServer for what could be a simple module
It can be tempting to use it to encapsulate state in a process, but sometimes it is definitely not the right tool for the job. Making a GenServer can create a bottleneck in your application. It will be a single actor handling all the calls from all over your application.
That one is most definitely on the list for a follow up edition as it’s my biggest pet peeve. Just limited myself to 10 and stayed with the “simpler basic data structures” for this one :)
If you want to encapsulate some shared state then (as far as I know) you’ve got to use a GenServer (or a wrapper around it like Agent), or you have to use some other process that can bottleneck, like a database or ETS.
Please correct me if I’m wrong about that!
The docs for GenServer and Agent are clear that you should be cautious about what processing you do in the caller’s process versus in the GenServer/Agent process, which I think should minimise this issue.
An ETS table isn’t a process, it’s a data structure. At this point, there are read_concurrency and write_concurrency modes that explicitly accomodate concurrent access from multiple processes.
Erlang and Elixir don’t disallow globally-accessible state, but they do make you think a little harder about it than most other languages do. :)
And yep, ETS isn’t a process but it can probably still be a bottleneck in the same way an external database can be, depending on your access patterns and settings, but I imagine it’s probably pretty fast because (as I understand it) it doesn’t do much work and has granular locking.
ETS isn’t a process but it can probably still be a bottleneck in the same way an external database can be
Ah no, not really. It’s generally safe (in my observation) to assume that ETS isn’t going to be your bottleneck unless you use it really and deliberately poorly. Always benchmark, sure, but you can basically count on ETS. DETS, Mnesia…those can be a different story of course.
Another option for global state is persistent terms, which are finicky in their usage but still quite handy.
Of course if you need to share state, that state has to be held by a process, and then Agent/GenServer/… are your friend.
What I was talking about was more about the beginner’s urge to put everything in a process, even when it does not need to. If the module is stateless, or that state is not shared, you don’t need a GenServer, and doing so can introduce avoidable bottlenecks. When you got a hammer, everything looks like a nail ;)
I think Elixir is a cool language, and I don’t regret learning it, but there are a bunch of gotchas and I have not been totally satisfied with Phoenix, which is a web framework written in Elixir that I learnt the language for.
Of these 10:
Ordering comparisons being broken has annoyed me a few times and seems totally avoidable. I think Elixir should have broken from Erlang here and defined ordering comparisons only between types that have a clear order (probably make it an interface any type can implement?)
I’ve never been bitten by Keyword lists being difficult to match on, but it is annoying. The fact that they’re multidicts rather than dicts also feels like it invites errors, because almost all the time you don’t want multiple values for a key and won’t have prepared for that, and I never remember whether Keyword.get fetches the first or last value for a key.
Some of my own gotchas/friction points with the language:
Nils bite me. I don’t like how many functions and operations return nil rather than erroring. E.g. x[some_invalid_key] is nil rather than an error.
Elixir (like many recent-ish languages) seems torn between errors as values and exceptions, and so there is a little friction working that out each time I’m using an unfamiliar function that can fail or when I’m writing a function that can fail.
use Vs import Vs alias is a bit obscure and took me a little while to understand. I think that something more uniform and a little more verbose would have been better.
with can be quite awkward compared to early returns
the parser can sometimes be fussy about where you put comments or about other things that I don’t expect to matter (iirc, if you have several lines between your with or for keyword and the do, then the parser can get confused if you comment out some of the lines (but not others))
too many macros when functions would have been fine (the phoenix router is bad for this, imo)
I don’t like that applying a macro is visually indistinguishable from applying (calling) a function
the standard library is fairly complete and has very good documentation, but the wider ecosystem is not so good: lots of stuff that you’d find in more established languages is either missing or done poorly (bad or incomplete implementation or documentation)
These ones are mostly my fault, and maybe off-topic but I oversold myself on phoenix and phoenix live-view:
I hoped live-view would let me use one language for the whole application and so avoid the friction of switching languages and tooling, but it didn’t work out like that.
live-view controllers and non-live controllers in phoenix are different enough that there is quite a lot of friction switching between them
doing network round-trips for stuff that doesn’t need it causes a lot of weird behaviours and bad UX that don’t show up when developing (because latency is low) but do show up in production. We’ve ended up doing some front-end work in JS and some in live-view.
it’s hard to access session data (data attached to a cookie) from the phoenix side, so sometimes you need a non-live controller basically just for that
have to learn a strange and under-documented DSL of HTML attributes etc. that control what live view does
CSS is still a bit pain and requires you to use another language and sometimes structure your HTML in weird ways
writing HTML is a drag because of the verbosity of the tags and stuff. I initially used phoenix_slim, so that my templates could be written in slim rather than Eex (Eex is pretty similar to ERB or PHP templates), but live-view and phoenix now use a format called Heex, which is not yet supported by phoenix_slim, and even if phoenix_slim is fixed, then the recommended generators and formatters etc won’t know about it.
Some gripes with phoenix:
phoenix wasn’t as stable as I’d like and has gone through several quite large changes particularly in how views are written and how they expect you to style things (they moved to tailwind). They’re careful to make this stuff opt-in, but it is still a pain to end up with a codebase (and online documentation, examples, etc) with essentially two different DSLs for writing frontend components and quite a few changes to how live-view works.
the provided form components are missing some important ones (e.g. multi select with check boxes, radio-button group) that I didn’t enjoy writing myself
generated admin views are much more work and less nice than Django’s
quite a lot of churn on how to handle CSS
the stuff about immutable state doesn’t actually help as much as you might think because all important state is held in (typically) a postgres database, so accessing and mutating state is still based on your understanding of SQL and transactions, not based on elixir.
If I were starting a new webapp (which I’d rather not!) then I might not pick Phoenix and instead put up with my dislike of JavaScript/TypeScript and go for supabase + svelte, or sveltekit. Or maybe I’d go for django-wagtail plus some typescript.
I do think Phoenix is an overly complex piece of technology, and I sincerely despise the generators it provides.
In the end, I like the simplicity of HTMX over LiveView. I generally only use Phoenix.Router (which is a nice addition to Plug) and Phoenix.Template (which they recommend not using directly, but oh well). HTMX has extensions for Server Sent Events and Websocket, and Hyperscript (which I also enjoy for the bits of scripting that are sometimes needed) does support those as well. The end result is far less boilerplate code, far less complexity, but you do have to know how Phoenix work to set it up that way, or to strip down everything you don’t need from the result of the generators.
If I need something more complex on the client side, I’ll go with a frontend framework and an OpenAPI which can be achieved with Plug alone (though, Phoenix.Router is still a nice addition). Or back when I was into the hype of GraphQL, I’d use absinthe (not anymore, too complex for the use cases I had, my data never looks like the kind of graphs that make GraphQL worth it).
I also really don’t like Ecto, mainly for petty reasons I admit. My main nitpick with it is that, contrary to Django’s ORM, you can’t switch backends via configuration. I want to use SQLite in dev and PostgreSQL in prod, and thus not require the dev to setup a local instance of PostgreSQL (even though it can be achieved via docker quite easily). In the test suite I even use sqlite://:memory:, which makes the CI faster (I can even include that test suite in a stage of a Dockerfile so that the image can only be built if the test suite is green). I can’t do that with Ecto without duplicating a lot of code and introducing a lot of complexity, shame.
I often use Elixir as some sort of “orchestrator” and will not hesitate to delegate tasks to other languages. Distributing some small Go/Rust/C binaries in your release that you interact with via stdin/stdout and JSON works wonders (kind of like a microservice without the network layer). This is also how I overcome the shortcomings of the ecosystem sometimes, without needing to write a NIF (which can be complex). Those binaries are also easily testable. Automating the build/release/deployment of those binaries is nothing a Makefile cannot do in a few lines.
My main nitpick with it is that, contrary to Django’s ORM, you can’t switch backends via configuration. I want to use SQLite in dev and PostgreSQL in prod
Been there and I was bitten in the arse by that, hard. Since then I always use the same DB in dev and prod, especially as it allows me to use my DB of choice up to its full potential and not just as a dumb storage. CTEs, different aggregate functions, indices that suits desirable use case, etc. All of that makes difference. When you use different DBs in dev and prod then you need to use lowest common denominator which quite substantially limits what you can do.
95% of my use cases are very basic dumb CRUD APIs that do not need CTEs, aggregate functions (SQLite still have some basic aggregate functions by the way), views, triggers, or PostgreSQL’s pub/sub.
Of course, if the need for this arise, I will ditch SQLite for dev, but only then. There is no valid reason to complexify needlessly the dev setup.
At the point where I need more than what SQLite has to offer and need to use more complex features, I also probably outgrown the ORM and will have basic tables to be fed by the Python side, with triggers that will call more complex SQL functions and populate other tables that will be aggregated in views, used by unmanaged Django models to hydrate Python objects (in order to close the loop).
When I do have such an architecture, of course I cannot rely on SQLite anymore on the dev’s machine and will setup a docker-compose stack or something. But that’s what? less than 5% of my use cases?
Heyo, thansk for your thoughts and they make sense and some stuff is on the list for the next post (i.e. alias vs. the rest):
lots of stuff that you’d find in more established languages is either missing or done poorly (bad or incomplete implementation or documentation)
What do you have in mind? I can think of fairly few for elixir my current list off the top of my head is basically:
better Admin generation/easier admin backends (you named this as well)
Kafka support is reportedly bad
when companies publish SDKs for their services, we usually don’t get one and so we gotta build one ourselves etc.
But other than that I think it’s very solid and cool so I’d like to know what you’re missing. I get the thing about slim, but honestly I don’t think it’s happening. The last commit was 3 years ago and I’m surprised it was this “recent” as I felt it wasn’t really maintained for a long time
when companies publish SDKs for their services, we usually don’t get one and so we gotta build one ourselves etc.
I made myself a small framework in Go, Rust and Python to write small JSON APIs with stdin/stdout as the transport layer (I did not publish it… yet?). And then use https://github.com/jayjun/rambo to encapsulate those binaries in an Elixir API (and eventually a GenServer if the binary must be long-running with some state). I know I could use NIFs, but this solution is far less complex and far easier to test.
This is how I overcome the shortcomings of the ecosystem.
You’re welcome, thanks for engaging with my message, I felt kinda silly for having spent a while writing it.
What do you have in mind?
Obviously my overwhelming feeling towards people who publish open source packages is gratitude and solidarity, so these aren’t judgements of the authors.
The most common missing thing is that I want an easy interface for some web service or whatever, which is partly companies not publishing SDKs as you say. I’ve also missed admin panels; component libraries for phoenix (some exist now, but phoenix frontend stuff has been quite unstable); and some more obscure stuff like counting STV votes (OpenTally does this in python).
I think a lot of packages have quite poor documentation (e.g. Recaptcha, Earmark (try to use the postprocessor option to as_html; try to signal an error from a postprocessor)) and others have been kind of unstable (incl phoenix-slim, etc). There were some with (IMO) poor implementations as well, but I’d rather not name them.
Yes, I agree that phoenix-slim looks unlikely to be revived. I think I should probably just configure my editor to make editing HTML less annoying.
Thanks for replying! Yeah that’s more or less what I had in mind, I had forgotten component libraries though - nice one! Was wondering if I had any other major blind spots, I’ve been thinking about writing another blog post where talking about Elixir’s eco system would be a part of it so another perspective here is hugely helpful!
Also, I often feel silly writing my long blog posts - when one generates so much discussion as this one I’m quite happy :D
Elixir (like many recent-ish languages) seems torn between errors as values and exceptions…
Yeah, I think this is actually something it inherits from Erlang, which is a product of the 80’s and 90’s when exceptions were Amazeballs New Hot Tech and people didn’t yet have a good feel for the various ways they made life harder. I suspect that if Elixir didn’t have to be able to handle Erlang exceptions then it might not have them.
A lot of your gotchas mirror my own, I wrote a post on them a while back called “Elixir Nitpicks”. :-P You sound like you have a lot more depth in Phoenix and Liveview than I do though.
It’s missing an 11th gotcha, which I think is quite important for beginners:
It can be tempting to use it to encapsulate state in a process, but sometimes it is definitely not the right tool for the job. Making a GenServer can create a bottleneck in your application. It will be a single actor handling all the calls from all over your application.
That one is most definitely on the list for a follow up edition as it’s my biggest pet peeve. Just limited myself to 10 and stayed with the “simpler basic data structures” for this one :)
If you want to encapsulate some shared state then (as far as I know) you’ve got to use a GenServer (or a wrapper around it like Agent), or you have to use some other process that can bottleneck, like a database or ETS.
Please correct me if I’m wrong about that!
The docs for GenServer and Agent are clear that you should be cautious about what processing you do in the caller’s process versus in the GenServer/Agent process, which I think should minimise this issue.
An ETS table isn’t a process, it’s a data structure. At this point, there are
read_concurrencyandwrite_concurrencymodes that explicitly accomodate concurrent access from multiple processes.Erlang and Elixir don’t disallow globally-accessible state, but they do make you think a little harder about it than most other languages do. :)
Thanks for the info about read_concurrency.
And yep, ETS isn’t a process but it can probably still be a bottleneck in the same way an external database can be, depending on your access patterns and settings, but I imagine it’s probably pretty fast because (as I understand it) it doesn’t do much work and has granular locking.
Ah no, not really. It’s generally safe (in my observation) to assume that ETS isn’t going to be your bottleneck unless you use it really and deliberately poorly. Always benchmark, sure, but you can basically count on ETS. DETS, Mnesia…those can be a different story of course.
Another option for global state is persistent terms, which are finicky in their usage but still quite handy.
Of course if you need to share state, that state has to be held by a process, and then Agent/GenServer/… are your friend.
What I was talking about was more about the beginner’s urge to put everything in a process, even when it does not need to. If the module is stateless, or that state is not shared, you don’t need a GenServer, and doing so can introduce avoidable bottlenecks. When you got a hammer, everything looks like a nail ;)
Cool, understood.
I started learning Elixir two days ago and even knew some of them. In my opinion, it’s still a cool language and still less weird than JavaScript.
It’s a great language - my favorite! Also cool on learning Elixir, it’s definitely fun and worth it!
I think Elixir is a cool language, and I don’t regret learning it, but there are a bunch of gotchas and I have not been totally satisfied with Phoenix, which is a web framework written in Elixir that I learnt the language for.
Of these 10:
Ordering comparisons being broken has annoyed me a few times and seems totally avoidable. I think Elixir should have broken from Erlang here and defined ordering comparisons only between types that have a clear order (probably make it an interface any type can implement?)
I’ve never been bitten by Keyword lists being difficult to match on, but it is annoying. The fact that they’re multidicts rather than dicts also feels like it invites errors, because almost all the time you don’t want multiple values for a key and won’t have prepared for that, and I never remember whether Keyword.get fetches the first or last value for a key.
Some of my own gotchas/friction points with the language:
Nils bite me. I don’t like how many functions and operations return nil rather than erroring. E.g.
x[some_invalid_key]is nil rather than an error.Elixir (like many recent-ish languages) seems torn between errors as values and exceptions, and so there is a little friction working that out each time I’m using an unfamiliar function that can fail or when I’m writing a function that can fail.
use Vs import Vs alias is a bit obscure and took me a little while to understand. I think that something more uniform and a little more verbose would have been better.
withcan be quite awkward compared to early returnsthe parser can sometimes be fussy about where you put comments or about other things that I don’t expect to matter (iirc, if you have several lines between your
withorforkeyword and thedo, then the parser can get confused if you comment out some of the lines (but not others))too many macros when functions would have been fine (the phoenix router is bad for this, imo)
I don’t like that applying a macro is visually indistinguishable from applying (calling) a function
the standard library is fairly complete and has very good documentation, but the wider ecosystem is not so good: lots of stuff that you’d find in more established languages is either missing or done poorly (bad or incomplete implementation or documentation)
These ones are mostly my fault, and maybe off-topic but I oversold myself on phoenix and phoenix live-view:
I hoped live-view would let me use one language for the whole application and so avoid the friction of switching languages and tooling, but it didn’t work out like that.
live-view controllers and non-live controllers in phoenix are different enough that there is quite a lot of friction switching between them
doing network round-trips for stuff that doesn’t need it causes a lot of weird behaviours and bad UX that don’t show up when developing (because latency is low) but do show up in production. We’ve ended up doing some front-end work in JS and some in live-view.
it’s hard to access session data (data attached to a cookie) from the phoenix side, so sometimes you need a non-live controller basically just for that
have to learn a strange and under-documented DSL of HTML attributes etc. that control what live view does
CSS is still a bit pain and requires you to use another language and sometimes structure your HTML in weird ways
writing HTML is a drag because of the verbosity of the tags and stuff. I initially used phoenix_slim, so that my templates could be written in slim rather than Eex (Eex is pretty similar to ERB or PHP templates), but live-view and phoenix now use a format called Heex, which is not yet supported by phoenix_slim, and even if phoenix_slim is fixed, then the recommended generators and formatters etc won’t know about it.
Some gripes with phoenix:
If I were starting a new webapp (which I’d rather not!) then I might not pick Phoenix and instead put up with my dislike of JavaScript/TypeScript and go for supabase + svelte, or sveltekit. Or maybe I’d go for django-wagtail plus some typescript.
I do think Phoenix is an overly complex piece of technology, and I sincerely despise the generators it provides.
In the end, I like the simplicity of HTMX over LiveView. I generally only use Phoenix.Router (which is a nice addition to Plug) and Phoenix.Template (which they recommend not using directly, but oh well). HTMX has extensions for Server Sent Events and Websocket, and Hyperscript (which I also enjoy for the bits of scripting that are sometimes needed) does support those as well. The end result is far less boilerplate code, far less complexity, but you do have to know how Phoenix work to set it up that way, or to strip down everything you don’t need from the result of the generators.
If I need something more complex on the client side, I’ll go with a frontend framework and an OpenAPI which can be achieved with Plug alone (though, Phoenix.Router is still a nice addition). Or back when I was into the hype of GraphQL, I’d use absinthe (not anymore, too complex for the use cases I had, my data never looks like the kind of graphs that make GraphQL worth it).
I also really don’t like Ecto, mainly for petty reasons I admit. My main nitpick with it is that, contrary to Django’s ORM, you can’t switch backends via configuration. I want to use SQLite in dev and PostgreSQL in prod, and thus not require the dev to setup a local instance of PostgreSQL (even though it can be achieved via docker quite easily). In the test suite I even use
sqlite://:memory:, which makes the CI faster (I can even include that test suite in a stage of a Dockerfile so that the image can only be built if the test suite is green). I can’t do that with Ecto without duplicating a lot of code and introducing a lot of complexity, shame.I often use Elixir as some sort of “orchestrator” and will not hesitate to delegate tasks to other languages. Distributing some small Go/Rust/C binaries in your release that you interact with via stdin/stdout and JSON works wonders (kind of like a microservice without the network layer). This is also how I overcome the shortcomings of the ecosystem sometimes, without needing to write a NIF (which can be complex). Those binaries are also easily testable. Automating the build/release/deployment of those binaries is nothing a Makefile cannot do in a few lines.
I believe that is possible!
Elixir makes an excellent control plane.
The solution is so simple. Lovely.
I always forget that you can put Elixir code in the body of a module to be executed at compile time.
Been there and I was bitten in the arse by that, hard. Since then I always use the same DB in dev and prod, especially as it allows me to use my DB of choice up to its full potential and not just as a dumb storage. CTEs, different aggregate functions, indices that suits desirable use case, etc. All of that makes difference. When you use different DBs in dev and prod then you need to use lowest common denominator which quite substantially limits what you can do.
95% of my use cases are very basic dumb CRUD APIs that do not need CTEs, aggregate functions (SQLite still have some basic aggregate functions by the way), views, triggers, or PostgreSQL’s pub/sub.
Of course, if the need for this arise, I will ditch SQLite for dev, but only then. There is no valid reason to complexify needlessly the dev setup.
At the point where I need more than what SQLite has to offer and need to use more complex features, I also probably outgrown the ORM and will have basic tables to be fed by the Python side, with triggers that will call more complex SQL functions and populate other tables that will be aggregated in views, used by unmanaged Django models to hydrate Python objects (in order to close the loop).
When I do have such an architecture, of course I cannot rely on SQLite anymore on the dev’s machine and will setup a docker-compose stack or something. But that’s what? less than 5% of my use cases?
Heyo, thansk for your thoughts and they make sense and some stuff is on the list for the next post (i.e. alias vs. the rest):
What do you have in mind? I can think of fairly few for elixir my current list off the top of my head is basically:
But other than that I think it’s very solid and cool so I’d like to know what you’re missing. I get the thing about slim, but honestly I don’t think it’s happening. The last commit was 3 years ago and I’m surprised it was this “recent” as I felt it wasn’t really maintained for a long time
I made myself a small framework in Go, Rust and Python to write small JSON APIs with stdin/stdout as the transport layer (I did not publish it… yet?). And then use https://github.com/jayjun/rambo to encapsulate those binaries in an Elixir API (and eventually a GenServer if the binary must be long-running with some state). I know I could use NIFs, but this solution is far less complex and far easier to test.
This is how I overcome the shortcomings of the ecosystem.
You’re welcome, thanks for engaging with my message, I felt kinda silly for having spent a while writing it.
Obviously my overwhelming feeling towards people who publish open source packages is gratitude and solidarity, so these aren’t judgements of the authors.
The most common missing thing is that I want an easy interface for some web service or whatever, which is partly companies not publishing SDKs as you say. I’ve also missed admin panels; component libraries for phoenix (some exist now, but phoenix frontend stuff has been quite unstable); and some more obscure stuff like counting STV votes (OpenTally does this in python).
I think a lot of packages have quite poor documentation (e.g. Recaptcha, Earmark (try to use the postprocessor option to
as_html; try to signal an error from a postprocessor)) and others have been kind of unstable (incl phoenix-slim, etc). There were some with (IMO) poor implementations as well, but I’d rather not name them.Yes, I agree that phoenix-slim looks unlikely to be revived. I think I should probably just configure my editor to make editing HTML less annoying.
Thanks for replying! Yeah that’s more or less what I had in mind, I had forgotten component libraries though - nice one! Was wondering if I had any other major blind spots, I’ve been thinking about writing another blog post where talking about Elixir’s eco system would be a part of it so another perspective here is hugely helpful!
Also, I often feel silly writing my long blog posts - when one generates so much discussion as this one I’m quite happy :D
Yeah, I think this is actually something it inherits from Erlang, which is a product of the 80’s and 90’s when exceptions were Amazeballs New Hot Tech and people didn’t yet have a good feel for the various ways they made life harder. I suspect that if Elixir didn’t have to be able to handle Erlang exceptions then it might not have them.
A lot of your gotchas mirror my own, I wrote a post on them a while back called “Elixir Nitpicks”. :-P You sound like you have a lot more depth in Phoenix and Liveview than I do though.
Which explains why they have their own on top? :P
I do like the convention to add a
!at the end of the function name to indicate that it might throw instead of returning an error, for example:Application.fetch_env(app, key)-> returns{:ok, val} | : errorApplication.fetch_env!(app, key)-> returnsvalor raises an exception