NIFs are rad. I know they’re often used for performance reasons but I really like them for exposing specific libs, or OS SDK functions (e.g. Core* stuff on MacOS) to Erlang code. NIF/BEAM provides a mechanism for keeping data around on the C side and a “reference” type that lets you wrap & return a C pointer and pass it around in your Erlang code, so that can be super nice - you can do all the allocation/release nitty-gritty neatly in your C code, and leave yourself free to just think about the application structure & flow in Erlang, which for me is one of the areas where it really shines. Also NIF provides a lot of “interface”-style stuff that (AFAICT) ports just don’t - when asking about interfacing with C code, Erlang people often say “just write a port and communicate over stdin/out”, which is great, but then you basically have to define your own comms mechanisms - whereas with NIF it’s just a function call and all the necessary methods for converting types are included.
Oh, maybe you (or some other lobster!) can help me with a thing. To the best of my knowledge, Erlang (and hence Elixir) use bignums for integers…unfortunately, the docs don’t say anything about how to get that across over in C land. Any ideas?
It doesn’t appear as though the NIF interface gives you an easy way to get Erlang bignums into your NIF. The icky way to pass in a bigint might be to encode it as a binary or string and decode it in the NIF. If you’re interested, you could take a peek at big.c and big.h. There are functions big_to_bytes, which should return an array of digits, and big_to_double, which should return a double (if the bignum fits).
Sounds right. I’ve not had to share numbers bigger than 64-bit ints so I’ve just used enif_get_int64 (or enif_get_long, enif_get_uint64 etc) to do the conversion (and relevant enif_make_* the other way), and haven’t had any issues with that.
int ei_decode_bignum(const char *buf, int *index, mpz_t obj)
Decodes an integer in the binary format to a GMP mpz_t integer. To use this function, the ei library must be configured and compiled to use the GMP library.
Erlang is a safe, functional language with high reliability. The NIF’s can boost performance but break safety or other properties. This looks like a good, use case for one of the safe, systems languages we have now. The code still stays safe even when breaking Erlang model a bit for performance. There might even be a way to use advanced features to encode whatever structure is necessary to get them to play nice with the scheduler. That part is purely speculative, though.
NIFs are extremely useful, but they’re also a foot-gun with a hair trigger. In addition to being able to take down your Erlang node from a faulty NIF call, calling long-running NIFs (where long-running usually means >1ms) can have degenerate effects on VM performance.
Erlang scheduler threads are written to rapidly switch out Erlang processes and communicate with one another. When a scheduler runs a long-running NIF, that scheduler is no longer able to communicate with the other scheduler threads until the NIF finishes running. You can play around with this to get a better feel of how NIFs can misbehave and have large effects on the BEAM.
Please don’t take this as discouraging writing NIFs, though! If you are considering writing a NIF, please read this carefully. It will save you a lot of pain and misery down the road.
If memory serves, Erlang recently added better support for helping NIFs play nice with the scheduler. I haven’t had occasion to use that, though, since if I’m writing C I want it in a port for stability anyways. :)
NIFs are rad. I know they’re often used for performance reasons but I really like them for exposing specific libs, or OS SDK functions (e.g. Core* stuff on MacOS) to Erlang code. NIF/BEAM provides a mechanism for keeping data around on the C side and a “reference” type that lets you wrap & return a C pointer and pass it around in your Erlang code, so that can be super nice - you can do all the allocation/release nitty-gritty neatly in your C code, and leave yourself free to just think about the application structure & flow in Erlang, which for me is one of the areas where it really shines. Also NIF provides a lot of “interface”-style stuff that (AFAICT) ports just don’t - when asking about interfacing with C code, Erlang people often say “just write a port and communicate over stdin/out”, which is great, but then you basically have to define your own comms mechanisms - whereas with NIF it’s just a function call and all the necessary methods for converting types are included.
Oh, maybe you (or some other lobster!) can help me with a thing. To the best of my knowledge, Erlang (and hence Elixir) use bignums for integers…unfortunately, the docs don’t say anything about how to get that across over in C land. Any ideas?
It doesn’t appear as though the NIF interface gives you an easy way to get Erlang bignums into your NIF. The icky way to pass in a bigint might be to encode it as a binary or string and decode it in the NIF. If you’re interested, you could take a peek at big.c and big.h. There are functions
big_to_bytes, which should return an array of digits, andbig_to_double, which should return a double (if the bignum fits).Sounds right. I’ve not had to share numbers bigger than 64-bit ints so I’ve just used enif_get_int64 (or
enif_get_long,enif_get_uint64etc) to do the conversion (and relevantenif_make_*the other way), and haven’t had any issues with that.Does that work in C ports?
I don’t think so. I think you want to use ei functions like
ei_encode_*andei_decode_*for that.Aha, sure enough:
Thanks for the direction.
Erlang is a safe, functional language with high reliability. The NIF’s can boost performance but break safety or other properties. This looks like a good, use case for one of the safe, systems languages we have now. The code still stays safe even when breaking Erlang model a bit for performance. There might even be a way to use advanced features to encode whatever structure is necessary to get them to play nice with the scheduler. That part is purely speculative, though.
You might be interested to know that Rustler supports annotating Rust NIFs as dirty NIFs :)
That’s great! Exactly the kind of thing I was thinking of.
NIFs are extremely useful, but they’re also a foot-gun with a hair trigger. In addition to being able to take down your Erlang node from a faulty NIF call, calling long-running NIFs (where long-running usually means >1ms) can have degenerate effects on VM performance.
Erlang scheduler threads are written to rapidly switch out Erlang processes and communicate with one another. When a scheduler runs a long-running NIF, that scheduler is no longer able to communicate with the other scheduler threads until the NIF finishes running. You can play around with this to get a better feel of how NIFs can misbehave and have large effects on the BEAM.
Please don’t take this as discouraging writing NIFs, though! If you are considering writing a NIF, please read this carefully. It will save you a lot of pain and misery down the road.
If memory serves, Erlang recently added better support for helping NIFs play nice with the scheduler. I haven’t had occasion to use that, though, since if I’m writing C I want it in a port for stability anyways. :)