The combination of languages/runtimes. A pure functional distributed message passing system for communication, and a strongly-typed, safe, and fairly low-level language for heavy number crunching.
got it. thank you. yes – I agree a powerful combination (although I probably would vote for a language with stronger shape-based (not declaration based) type system).
Erlang’s OTP and folks who designed where true visionaries of application development paradigms.
I really like these stories if refactoring, they’re like contextualized programming pearls. I remember on here a while ago a similar kind and of article on sublime text and mmap. It’d be cool to compile into a sort of “architecture of proprietary software” haha
There’s quite a bit of proprietary software I know have interesting architecture: some documented well like Windows NT (yes! it’s actually solid under the hood) and less documented, like Unreal.
Something along the lines of what @puhrez replied. However, I don’t think the problem (fast set/list insertion/removal operation on a large data structure) is trivial. What sucks is that people who could implement this, work for a company that makes web chat.
It seems that he’s commenting on the perceived triviality of the problem vis a vis the expertise assigned to it. i.e. member list vs aerospace modelling
“..
Today, the Rust backed SortedSet powers every single Discord guild: from the 3 person guild planning a trip to Japan to 200,000 people enjoying the latest, fun gam ..”
Perhaps I do not appreciate the scale of the problem
But erlang has ets – which is distributed in memory key value store, that can be sorted.
Some time ago, when I looked at it, it was backed up by efficient C-based data structure called
‘Juddy array’.
It was optimized for cache locality, and ETS data was not garbage collected (this combo allowed most recently added entries to be closest to CPU registers)
What was slow about it, is sorting/ordering (from what I remember)
In just recent release of Erlang OTP 22, substantial performance improvements were made to ordered ETS
http://www.erlang.org/news/132
“..ETS option write_concurrency now also effects and improves scalability of ordered_set tables. .. “
I guess, would be interesting to know why ETS (especially one from OTP 22) would not fit the bill (if developers would look at the problem with OTP 22 release)
This is an interesting read, but additional business context explaining the performance requirements would help readers make sense of some of the decisions. I’ve never used this product so it’s unclear to me why 640μs or even 19ms in the worst case for inserting members into these lists is not fast enough. Also, why does this list need to be sorted. Why would a map not work for this usecase?
Finally the future is starting to look like what it should look like.
are you referring to a combination of languages/runtime systems? (that is Elixir+Rust, and Erlang OTP+Rust runtime)?
Or you meant the feature / performance of the Discord service?
The combination of languages/runtimes. A pure functional distributed message passing system for communication, and a strongly-typed, safe, and fairly low-level language for heavy number crunching.
got it. thank you. yes – I agree a powerful combination (although I probably would vote for a language with stronger shape-based (not declaration based) type system). Erlang’s OTP and folks who designed where true visionaries of application development paradigms.
I really like these stories if refactoring, they’re like contextualized programming pearls. I remember on here a while ago a similar kind and of article on sublime text and mmap. It’d be cool to compile into a sort of “architecture of proprietary software” haha
There’s quite a bit of proprietary software I know have interesting architecture: some documented well like Windows NT (yes! it’s actually solid under the hood) and less documented, like Unreal.
I stole the the idea from the book the architecture of open source applications, but it’d definitely be cool to see into those worlds
The code is cool. The reasoning is… weird:
It’s a pity that such smart programmers have such big projects.
What do you mean?
Something along the lines of what @puhrez replied. However, I don’t think the problem (fast set/list insertion/removal operation on a large data structure) is trivial. What sucks is that people who could implement this, work for a company that makes web chat.
If I were a smart programmer, I’d be delighted to implement an efficiency improvement like this at a scale like this, regardless of the domain.
I don’t think that making tools that people enjoy is unfulfilling, especially if you enjoy gaming etc.
It seems that he’s commenting on the perceived triviality of the problem vis a vis the expertise assigned to it. i.e. member list vs aerospace modelling
Perhaps I do not appreciate the scale of the problem But erlang has ets – which is distributed in memory key value store, that can be sorted. Some time ago, when I looked at it, it was backed up by efficient C-based data structure called ‘Juddy array’. It was optimized for cache locality, and ETS data was not garbage collected (this combo allowed most recently added entries to be closest to CPU registers)
What was slow about it, is sorting/ordering (from what I remember)
In just recent release of Erlang OTP 22, substantial performance improvements were made to ordered ETS http://www.erlang.org/news/132
I guess, would be interesting to know why ETS (especially one from OTP 22) would not fit the bill (if developers would look at the problem with OTP 22 release)
This is an interesting read, but additional business context explaining the performance requirements would help readers make sense of some of the decisions. I’ve never used this product so it’s unclear to me why 640μs or even 19ms in the worst case for inserting members into these lists is not fast enough. Also, why does this list need to be sorted. Why would a map not work for this usecase?