This article is a bit of a rambling rant about threads, setting up a bunch of dubious strawmen, and doesn’t say:
Honestly the title feels like clickbait given how low little it’s related to the article text.
Thanks for the feedback.
It wasn’t really supposed to be about keydb - that was just used as an example of what would work better (versus redis) - the links and such are edits dzone did. KeyDB was used as an example of showing what can/should be done for future facing software.
Interesting! I wrote an entire article without saying that word - I didn’t plan that - the premise here is that a lot of older nix software (eg: let’s pick on postgresql) is old and not written to excel in a single process (but many threaded) system. I could’ve easily picked apart postgres instead but the keydb/redis was much easier to illustrate the point. Not only that, but developers have been pushed not to advance this style of architecture. My argument here is that if people would adopt this over something that embraces sys-v ipc style we could open up a much better ecosystem.
There was work even way back when on making concurrency and/or parallelism easier to get right. Here’s some for you to check out in case you find them interesting: Concurrent Pascal (among others), Eiffel’s SCOOP, Clay’s type-safe locks (pdf), ParaSail, and DTHREADS (pdf).
Since you mentioned Python, I’ll throw in that a few people told me they just used GNU Parallel with their scripts. I’m not sure how they do the integration step if there is one. I imaging pulling stuff out of text files into collections is common for them. GNU Parallel solves the harder problem with a one-liner in the terminal.
In an unikernel context Parallel wouldn’t be an option. It solves the main job of executing N things yet still in the old create a new process way - that’s what I’m arguing to get rid of because of it’s numerous security and performance issues. Python has many non-trivial limitations (beyond the GIL) that would need to be addressed to have true multithreading. Jython kinda meets in the middle but introduces other issues. I’m not saying any of this is easy - just something to aspire to.
Going back to how all this relates to unikernels - many unikernels have support for multiple threads but not multiple processes by design. One of the things that needs to happen to make the ecosystem thrive is more keydbs and less redis’.
Honestly, I didn’t get to the unikernel claims since the article was pretty ranty on the anti-Python, anti-Guido stuff. I’d have made it if you just did a technical breakdown of what older solutions did, what modern hardware does, how it doesn’t fit, and what you were promoting. Those kinds of articles also get more support in general on Lobsters. More potential uptake, too. I agree Parallel itself wouldn’t fit a unikernel but a Parallel syscall might.
It didn’t really come off as anti-python and/or Guido to me, but that might just be the last year or two giving me a higher bar for “anti Guido” :(
I don’t care for Python, but Guido always struck me as a really thoughtful, nice guy; those nerds who drove him out of Python should be tarred and feathered.
for the record I’m neither anti-Guido nor anti-python - that section was really intended to highlight:
a) the inherent struggle to take advantage of native threads in interpreted languages such as python (it affects most, all? of them)
b) the time period that certain views get espoused (1998, 2003, even up to 2008) these arguments had technical merit but the advent of real good threads in linux in ’03 and the drive towards many cores/many socket machines during those years really removed those technical arguments imo
as evidenced in the prior section Go did not suffer this same fate even though one or more of it’s creators had similar arguments for the ability of the average developer to take advantage of threads to the extent that they designed one of it’s core features goroutines as an abstraction around them
Performance of fork() I can buy, but insecurity? That I do not. In a multithreaded program, one bad reference and the poof! There goes the entire program, but in a multiprocess program, only one process goes poof on a bad reference (remember my days of programming on the Amiga sans MMU). Also, there’s been studies that have down fine grained locking degrades performance just as much as a single global lock (perhaps even more). Also, multithreaded programming is hard to reason about, which is why I personally prefer the actor model (share nothing message passing) over accessing shared mutating state.
insecurity for this definition means ‘thwarting bad actors’ not making your program crash from a bad memory reference due to programming error - i’d put that in the memory safety camp; also this is in the unikernel (single process) context not necessarily a linux system
being constrained to whatever you can do in the memory layout of a single executable is quite different than being able to execute random other arbitrary programs because your system is inherently built for that (eg: your average monero cryptominer bot dropper is most assuredly going to come in binary format and prob. written in a diff. language); your average shellcode does just that - pops a shell; for example i’ve yet to see an entire mysql client written in pure shellcode w/no calls to fork/exec*