Some really neat ideas, but also, a lot of headshaking around things like the XLA/Tensorflow dependency. The serious numerical story in Elixir doesn’t exist because core doesn’t care and, frankly, most Elixir folks are basically just doing web stuff as Ruby + Rails refugees.
Nice agitprop though if you want to boost your language’s popularity by riding the AI/ML gravy train.
BEAM would not be my first choice for anything where I had to do some serious number crunching. I wonder who will even use this, and if it’ll die out like the Swift extensions for it.
I use Elixir because it’s good at networking, concurrency, and parsing, not anything AI.
I had the same initial reaction and would rather bet on Julia for the future of ML. But at the very least, breaking Python’s monopoly is good and, of course, there is no good reason for Python’s ascent in this field. So, there’s no reason other ecosystems that have certain advantages over Python shouldn’t try to make a dent here.
A lot of ML projects use Ray (ray.io) for orchestration and workflows (or so I hear), basically implementing the actor model, and Elixir/BEAM is probably a more natural fit for that side of the problem.
Seems like they are using multistage programming to make this rather fast - I’m guessing this stuff wouldn’t be running on the BEAM directly? Could be wrong though.
The serious numerical story in Elixir doesn’t exist because core doesn’t care
Ehhhh. Clearly you don’t mean “Elixir core”, as Jose is the one writing this. Even if you mean the Erlang core team, they might not care enough to lead implementation but they’re open to additions that support it: https://github.com/erlang/otp/pull/2890
Anyway, BEAM isn’t the logical execution environment for this stuff, it needs GPU to fly. For this use case, BEAM is a great place to write a compiler, and will function adequately as a test environment.
FWIW, the compiler backend of Nx is pluggable, so other compilers aside from XLA can also be integrated with. XLA happened to be the first one they’ve gone with thus far.
Paraphrasing a colleague’s gripes…Elixir has a bunch of math functionality inherited from Erlang, and hasn’t bothered to fix any of the problems around it.
In erlang:
2> math:exp(344).
2.4963287283217065e149
3> math:exp(3444).
** exception error: an error occurred when evaluating an arithmetic expression
in function math:exp/1
called as math:exp(3444)
In Elixir:
iex(3)> :math.exp(344)
2.4963287283217065e149
iex(3)> :math.exp(3444)
** (ArithmeticError) bad argument in arithmetic expression
(stdlib 3.13.2) :math.exp(3444)
What’s going on here is that Erlang uses doubles internally (except for integers, but that’s a different kettle of fish), but does not do IEEE754 support for special values like NaN or Infinite. This is certainly a choice they could make, but it rears its ugly head when you implement something mathematically well-behaved like the sigmoid function:
iex(5)> sigmoid = fn (x) -> 1.0 / (1.0 + :math.exp(-x)) end
#Function<44.97283095/1 in :erl_eval.expr/5>
iex(6)> sigmoid.(-1000)
** (ArithmeticError) bad argument in arithmetic expression
(stdlib 3.13.2) :math.exp(1000)
iex(6)> sigmoid.(-100)
3.7200759760208356e-44
In JS, we’d get a negative in the denominator, it’d go to Inf, and the function would work fine. In Elixir, it explodes.
Core has had the opportunity to fix this for a long time, but has been spending cycles elsewhere.
Not really, as most of “interesting” part of IEEE is already there. What is not supported are qNaN (sNaN is supported as it’s behaviour is exactly the same as what Erlang does - just throw exception) and infinites. All other values are fully supported in Erlang. And while it can be irritating to have exception in infinity, I think that in most situations you do not mind, as that would mean that the result of computation would be probably garbage to you anyway. So throwing up early is probably what will provide clearer information, especially with Erlang error isolation approach.
The serious numerical story in Elixir doesn’t exist because core doesn’t care and, frankly, most Elixir folks are basically just doing web stuff
That perfectly describes how I got into Elixir. Phoenix got me into the ecosystem, but I branched out pretty quickly once I understood the power of the OTP model. About two years ago I started looking into writing drone flight control software in Elixir and ran into a huge hurdle with the numerical processing portion of it; I was fiddling around with NIFs and things, but it felt like I was just compromising the robustness of the system by having it call out to my own C code.
The Nx project has me very very excited that the project I put on the shelf might be viable now!
Julia might be better inspiration; I hope they didn’t stop with the Python ecosystem.
You might want to elaborate why this might be :)
Some really neat ideas, but also, a lot of headshaking around things like the XLA/Tensorflow dependency. The serious numerical story in Elixir doesn’t exist because core doesn’t care and, frankly, most Elixir folks are basically just doing web stuff as Ruby + Rails refugees.
Nice agitprop though if you want to boost your language’s popularity by riding the AI/ML gravy train.
Same applied to Python way back. That the Elixir devs are attempting to broaden their ecosystem is good.
BEAM would not be my first choice for anything where I had to do some serious number crunching. I wonder who will even use this, and if it’ll die out like the Swift extensions for it.
I use Elixir because it’s good at networking, concurrency, and parsing, not anything AI.
I had the same initial reaction and would rather bet on Julia for the future of ML. But at the very least, breaking Python’s monopoly is good and, of course, there is no good reason for Python’s ascent in this field. So, there’s no reason other ecosystems that have certain advantages over Python shouldn’t try to make a dent here.
A lot of ML projects use Ray (ray.io) for orchestration and workflows (or so I hear), basically implementing the actor model, and Elixir/BEAM is probably a more natural fit for that side of the problem.
On the other hand, not everybody needs serious number crunching. This could let people in the elixir ecosystem stay within it for longer.
Seems like they are using multistage programming to make this rather fast - I’m guessing this stuff wouldn’t be running on the BEAM directly? Could be wrong though.
Ehhhh. Clearly you don’t mean “Elixir core”, as Jose is the one writing this. Even if you mean the Erlang core team, they might not care enough to lead implementation but they’re open to additions that support it: https://github.com/erlang/otp/pull/2890
Anyway, BEAM isn’t the logical execution environment for this stuff, it needs GPU to fly. For this use case, BEAM is a great place to write a compiler, and will function adequately as a test environment.
¯\_(ツ)_/¯FWIW, the compiler backend of Nx is pluggable, so other compilers aside from XLA can also be integrated with. XLA happened to be the first one they’ve gone with thus far.
What do you mean with “the core doesn’t care”?
Paraphrasing a colleague’s gripes…Elixir has a bunch of math functionality inherited from Erlang, and hasn’t bothered to fix any of the problems around it.
In erlang:
In Elixir:
Contrast with JS:
What’s going on here is that Erlang uses doubles internally (except for integers, but that’s a different kettle of fish), but does not do IEEE754 support for special values like
NaNorInfinite. This is certainly a choice they could make, but it rears its ugly head when you implement something mathematically well-behaved like the sigmoid function:In JS, we’d get a negative in the denominator, it’d go to
Inf, and the function would work fine. In Elixir, it explodes.Core has had the opportunity to fix this for a long time, but has been spending cycles elsewhere.
I found no tickets in the Erlang (where BEAM issues belong) or Elixir bug trackers discussing infinity.
For all the supposed number crunching going on, nobody’s requesting working IEEE doubles?
Not really, as most of “interesting” part of IEEE is already there. What is not supported are
qNaN(sNaNis supported as it’s behaviour is exactly the same as what Erlang does - just throw exception) and infinites. All other values are fully supported in Erlang. And while it can be irritating to have exception in infinity, I think that in most situations you do not mind, as that would mean that the result of computation would be probably garbage to you anyway. So throwing up early is probably what will provide clearer information, especially with Erlang error isolation approach.That perfectly describes how I got into Elixir. Phoenix got me into the ecosystem, but I branched out pretty quickly once I understood the power of the OTP model. About two years ago I started looking into writing drone flight control software in Elixir and ran into a huge hurdle with the numerical processing portion of it; I was fiddling around with NIFs and things, but it felt like I was just compromising the robustness of the system by having it call out to my own C code.
The Nx project has me very very excited that the project I put on the shelf might be viable now!