This is an old prototype. I ended up making a language for it from scratch so that I could attach provenance metadata to values, making them directly editable even when far removed from their original source.
In Lil, a scripting language which includes the concepts of a database-style table and a query language for it, I might tackle tackle this problem something like the following:
txs:insert date amount text with
"2022-05-13T11:01:56.532-00:00" -3.30 "Card transaction of 3.30 CAD issued by Milano Coffee Roasters VANCOUVER"
"2022-05-12T10:41:56.843-00:00" -3.30 "Card transaction of 3.30 CAD issued by Milano Coffee Roasters VANCOUVER"
"2022-05-12T00:01:03.264-00:00" -72.79 "Card transaction of 72.79 CAD issued by Amazon.ca AMAZON.CA"
"2022-05-10T10:33:04.011-00:00" -20.00 "e-Transfer to: John Smith"
"2022-05-11T17:12:43.098-00:00" -90.00 "Card transaction of 90.00 CAD issued by Range Physiotherapy VANCOUVER"
end
date_tags:raze insert k v with
"2022-05-12T00:01:03.264-00:00" "things"
end
text_tags:raze insert k v with
"*Coffee*" "eating-out"
"*Range Physio*" "medical"
end
txs:update tag:date_tags@date where date in date_tags from txs
each v k in text_tags
txs:update tag:v where text like k from txs
end
print["untagged"]
show[select where !tag from txs]
print["totals by tag"]
show[select tag:first tag total:sum amount by tag where tag from txs]
But Lil isn’t just a scripting language; it’s also part of the Decker ecosystem, which has its own facilities for representing tables as manipulable and automatically serialized “grids”. A Decker-based user interface might contain a script something like this to compute the two output tables from the inputs:
on view do
txs:transactions.value
date_tags:raze dates.value
text_tags:raze texts.value
txs:update tag:date_tags@date where date in date_tags from txs
each v k in text_tags
txs:update tag:v where text like k from txs
end
untagged.value:select where !tag from txs
totals.value:select tag:first tag total:sum amount by tag where tag from txs
end
In the top example, i’m using Lil’s insert query form; a table literal much like your list-of-dicts in the Clojure version. It’s possible to hardcode tables like that in scripts within the Decker environment, but it’s more common for tables to be “stored” in grid widgets. The data within a grid can be directly edited:
I think this is still missing the point. If I’m looking at a row in the “untagged” view that you created, how do I add a tag to it? My goal was that you just directly edit the tag, and that flows back to the underlying data (demos in https://lobste.rs/s/jcsfbx/program_is_database_is_interface#c_i80teg). As opposed to having to navigate to the source table and scroll around for the row in question.
There are some previous research systems that do this using lenses, but I found that the lens laws are often violated by regular ui idioms and that just tracking per-value provenance was good enough most of the time.
Long post, many thoughts, but I don’t feel like doing a lot of editing, so apologies in front for
unfiltered feedback! I don’t mean the tone I will use here :-)
The start of the article is 🤌, but it sort of does’t live to my expectations. I feel this is
mostly about extensible, forward compatible enums, which is quite neat (I didn’t realize that “I
want to add a field to all these enum variants” admits such an elegant solution), but I don’t think
solves my problems with error handling facilities in languages.
Basically, this post feels like it attacks “make complicated things possible” part of the problem,
and, sure, if you add across and along non-exhaustiveness, !__ for warnings, auto-delegation to
turn an enum into struct, a capability system to track panics, you can solve everything.
But the problem I see with error handling is that we don’t know how to make simple things easy.
That’s especially true in Rust, of course, but it seems that in every language a simple way to go
about error handling leads to some pretty significant drawbacks, and the money question is not how
can we add extra knobs to handle all of the requirements, but whether there’s some simple idea
that kinda solves stuff?
Like, for example, the sled problem — we want every function to be
precise about its specific error conditions, but, in practice, the stable equilibrium is one error
type for library. Everr (fabulous name by the way, great job) suggest
The Everr language server has a code action for defining error types for a given function based on
the error types of the functions that are called. It also can intelligently suggest modifications to
error cases as the function body evolves over time, taking contextual rules such as access control
into consideration.
But this sucks! With a nominal type system, having to name every function, and every function’s
error type is very much not easy, and even if you add a bunch of tooling support, the result would
still not be easy.
Another simple-things-are-hard problem in error handling is exposing details. If you write a library
A, and it uses a library B, and B is an implementation detail, then a common API design pitfall is
to leak B through your error types (either directly, by including B variants, or indirectly, by
allowing downcasing to B). The problem here isn’t that it’s impossible to either expose or hide B
properly. There’s a bunch of techniques available for that (but I belive that Everr makes them nicer
and more powerful). The problem is that you need to decide what do you do, and that is hard. You
need pretty high level of discipline and experience to even note that this is a problem.
Or another common pitfall of type-based error types, where
That is, that the fact that you can aggregate errors based on types doesn’t mean that you should.
I have no idea how to handle errors in general! I just don’t have bullet proof recipes, every time
it is “well, let’s look at your specific situation, shall we?”. Super annoying!
I don’t think that a lack of language mechanisms is my problem. What I lack is a gang-of-four book
for patterns of error management (I mean, I have such a book at my head obviously, and I consult it
often when writing code, but I can’t condense it to a single-paragraph to put into project’s style
guide and call it a day).
Assorted smaller thoughts:
For an article about systems programming language, it is surprising that no space is dedicated to
the ABI. How exactly do you raise in catch errors, in terms of which bytes go into which register,
I feel is an unsolved problem. Returning values is allegedly slow. Unwinding is,
counterintuitively, faster (see Duffy’s post & the recent talk on C++ exceptions in embedded (has
anyone reproduced that result in particular?)). To avoid potential ambiguity: rust-style error
handling, and Java-style exceptions differ on two orthogonal axis:
whether you syntactically allocate expressions that throws (type system&syntax stuff)
whether throwing happens by stack unwinding (by the way, what is the best one-page explainer, of
how unwinding actually works? I am embarrassed to admit that unwinding is magic pixie dust for
me, and I have no idea how landing pads work), or by “normal” return.
I am strictly speaking about the second one.
And than, there’s Zig, and than there’s this paper by Sutter of several years ago which says that “actually, you do want to return an integer” to be fast.
heap exhaustion
Heap exhaustion is not the central example of OutOfMemory error. The central example is someone
passing you a malformed gif image whose declared size is 67PiB. That’s the sort of thing that you
need to be robust to, a single rouge allocation due to a bug/malformed input.
It would also be interesting to know what sub-fraction of that group has tests for the
out-of-memory error handling code path, and how good that test coverage is.
No these data here, but, anecdotally, eyeballing Zig the code that has both allocator parameter,
try, and defer/errdefer usually tends to reveal errors.
Zig and Odin are different from other languages here; allocators are passed down ~everywhere as
parameters
Such discipline is possible in C++, Rust and other languages to varying extents, but is less
common. Rust has an unstable allocator_api feature, where the discussion originally started in
A lint that prevents error values from being discarded using standard shorthands (e.g. _ = ), without an explicit annotation, such as a comment or a call to an earmarked function (to allow for ‘Find references’) etc.
I used to think the what Rust does, with must_use, is the right thing, and was hesitant of swift approach of requiring everything to be used. After using Zig, I am sold though, no need to other think this, a non-void function whose result is unused and is not _ = should be a compilation error. The amount of false positives is vanishingly small.
“actually, you do want to return an integer” to be fast.
https://mcyoung.xyz/2024/04/17/calling-convention/ had an interesting idea. Change the abi so that the error/success payloads of Result are passed as out parameters, and then just return the Ok/Err tag. That seems like it allows the best of both worlds - effectively automating the common pattern used in zig and making it type-safe.
Returning values is allegedly slow. Unwinding is, counterintuitively, faster
I think this is true in the common case where an error did not occur. Returning error information adds overhead to both the caller and callee, whereas catch/throw has the famous “zero overhead.” On the other hand, when an error does occur, unwinding the stack is significantly slower because a bunch of compiler-generated metadata has to be looked up and processed for each active stack frame.
the recent talk on C++ exceptions in embedded
The talk I watched (i don’t remember who gave it) was primarily about code size, not performance. The common wisdom being that using C++ exceptions bloats your code with all those compiler-generated tables annd extra code for running destructors during unwinding.
Hey Alex, thanks for taking the time to read and share your thoughts. I always appreciate reading your blog posts, so thank you for the specific feedback on this post.
That’s especially true in Rust, of course, but it seems that in every language a simple way to go about error handling leads to some pretty significant drawbacks, and the money question is not how can we add extra knobs to handle all of the requirements, but whether there’s some simple idea that kinda solves stuff?
It would be helpful to have an operational definition of “simple” here with one or two examples before I attempt to answer this. 😅
For example, if there is a guideline that by default, an error should not expose structure, and just expose an interface like:
trait RichDebug: Debug {
type Kind: Debug
fn kind(&self) -> Kind
fn metadata(&self, &mut debug::Metadata<_'>) // similar to fmt::Formatter, but creates something like a JSON object instead of a string
}
and the implementation for this were to be derived using a macro (or comptime machinery in Zig), would that be considered “simple”?
Like, for example, the sled problem
Thanks for linking that blog post, I hadn’t read it earlier. This point stands out to me in particular:
inside the sled codebase, internal systems were [..] relying on the same Error enum to signal success, expected failure, or fatal failure. It made the codebase a nightmare to work with. Dozens and dozens of bugs happened over years of development where the underlying issue boiled down to either accidentally using the try ? operator somewhere that a local error should have been handled, or by performing a partial pattern match that included an over-optimistic wildcard match.
This goes directly against the Rust conventions RFC, which recommends using panics for “catastrophic errors”. I’ve seen this similar tendency in Go codebases, where people will put every kind of error under error, even if it’s technically a serious invariant violation (like a bounds check failure, which does trigger a panic!).
Based on Duffy’s writing on Midori, it feels like a Midori programmer would probably be more likely to use “abandonment” (panic) than a Rust/Go programmer in this kind of situation, given the built-in Erlang-style fault-tolerant architecture.
we want every function to be precise about its specific error conditions, but, in practice, the stable equilibrium is one error type for library
Right, so with Everr’s type system, you could write your code as returning only MyLibraryError, and then the language server can refactor the functions which need specific error conditions to instead return MyLibraryError:union+[.Case1 | .Case2].
The central example is someone passing you a malformed gif image whose declared size is 67PiB. That’s the sort of thing that you need to be robust to, a single rouge allocation due to a bug/malformed input.
This is a fair criticism. In that section, I originally intended to describe a system for regions/different sub-heaps based on some of the research on Verona (and in that context, “heap exhaustion” would mean “this sub-heap is exhausted”, not “the process heap is quite high for the running system”), but then I punted on that because I didn’t feel confident in Verona’s system, so I moved that description to the appendix.
I will update this.
and was hesitant of swift approach of requiring everything to be used. After using Zig, I am sold though
I personally prefer Swift’s approach of warning instead of a hard error, given that iterating on code becomes more fiddly if you need to keep putting/removing _ (speaking from first-hand experience with Go).
However, the point you’ve quoted here is talking about something slightly different. It’s saying that using the same shorthand for discarding ordinary and discarding errors is itself error-prone. Discarding errors should require noisier syntax (in lint form), because an error being discarded is likely to carry higher risk than a success value being discarded.
I have such a book at my head obviously [..]
Perhaps a good idea for a IronBeetle episode? I’m slowly working my way through the list; maybe you’ve already covered this in one of them. 😄
For an article about systems programming language, it is surprising that no space is dedicated to the ABI. How exactly do you raise in catch errors, in terms of which bytes go into which register, I feel is an unsolved problem
I omitted this because:
Since this point is primarily about performance, it doesn’t make sense for me to speculate about designs without having concrete measurements. Performing any kind of realistic measurement would likely be a fair amount of work.
I didn’t realize it was an “unsolved problem” but rather my working assumption was that “for ABI, the small number of people working on it pretty much know what all their options are, so it doesn’t make sense for me to tell them that”. For example, if you only care about 64-bit machines, perhaps you’re fine with burning a register on errors specifically (like Swift). For larger errors, you could reuse the same register as an out-parameter (as described in mcyoung’s post linked by Jamie).
Thanks for organizing this event! It was a good use of time – the talks were interesting and well paced
I’d personally be biased toward more “experience reports” than “new ideas”, but I think it’s a good format, and maybe others can take some cues for a variety of events in the same format
I wouldn’t have minded some more organized discussion afterward, although I don’t have any ideas on how to make that happen
I’m also glad Zulip worked out, since TBH if it was on Slack or Discord I may not have joined :)
I wouldn’t have minded some more organized discussion afterward, although I don’t have any ideas on how to make that happen
Yeah, it’s a weird common knowledge problem. There could have been a big discussion the day afterwards if everyone believed there was going to be one. I have thought about spreading the talks out more to give the discussion more room to breathe. Maybe one block per evening or similar.
Zulip has a free tier where you’re on your own and several monthly-fee-per-user tiers that offer support. I wasn’t sure how many attendees would pay less than the zulip fee for their tickets, so I contacted zulip and we agreed to just pay $1000 flat fee for support.
Paying $1k to have rapid-response support from Zulip for a live conference sounds pretty reasonable.
Fwiw if I was running a non-profit conference I would have just done it on the free tier and it probably would have been fine. I just felt that if I’m charging money for tickets then I should also be paying for the platform.
I don’t understand how this is really any different than typical serialization.The recursion depth is only 1 deep, but that’s enough for you to still need to write the code that reflects all the fields and follows the pointers. He says you only need one writev call, but that would be true even with an unlimited number of pointers, the whole point of writev is that it can do an arbitrary amount of work. He says it makes it so the on disk format can be exactly the same, but then admits it actually can’t be because of the pointers for the arrays and hash tables, and if your serialization already has to be smart enough to handle that it’s only a tiny bit more work to handle to arbitrary depth. Also how do you do a hash table where the values are variable length arrays? If you try to express it as indexes you need a variable length array of variable length arrays, and then you can’t reduce that to arrays of indexes, somewhere you need an array of pointers. If in this scheme you’re allowed to nest hash tables and dynamic length arrays then you are long the problem of handling arbitrary depth and you’re back at square one. Am I missing something here?
that would be true even with an unlimited number of pointers
Maybe useful context is that this is how the incremental parts of the zig compiler is being written. It only needs to write/read O(1) chunks of memory to save/load the entire state of the compiler, whereas if it used internal pointers it would be O(amount of code being compiled).
that would be true even with an unlimited number of pointers
Zig hashmaps can have custom hash/comparison functions. So you can have one big array of data, a ‘slice’ type which is a offset+length into that array, and a hashmap that knows how to look up the data for those ‘slices’.
HashMap<K, Vec> would become (HashMap<K, OffsetAndLen>, Vec) as I explained above, giving a constant 2 regions of memory. Just like the string examples in the talk.
Having owned a Framework since April of 2022, I cannot recommend them to people who need even basic durability in their devices. Since then, I have done two mainboard replacements, two top cover replacements, a hinge replacement, a battery replacement, several glue jobs after the captive screw hubs sheared from the plastic backing…
It’s just such an absurdly fragile device with incredibly poor thermals. They sacrificed a ton of desirable features to make the laptop repairable, but ultimately have released a set of devices that, when used in real-world settings, end with you repairing the device more often than not. And these repairs are often non-trivial.
I will personally be migrating to another machine. The Framework 12’s focus on durability may be trending in the right direction, but to regain trust, I’d need to see things like drop and wear tests. A laptop that can be repaired, but needs constant upkeep/incredibly delicate handling, is ultimately not an actual consumer device, but a hobbyist device.
Maybe they’ll get better in a few years. Maybe the Framework 12 will be better. Their new focus on AI, the soldered RAM in the desktop offering, and the failure to address the flimsy plastic chassis innards, among other things, mean that they have a long way to go.
It’s definitely a “be part of the community that helps solve our product problems” sort of feeling.
I have an AMD FW13, and was trying to figure out why it loses 50+% of its battery charge overnight when I close the lid, because I don’t use this computer every single day and don’t want to have remember to charge yet another device.
So I check the basics-I’m running their officially supported Linux distro, BIOS is current, etc. And half an hour into reading forum threads about diagnosing sleep power draw, I realize that this is not how I want to spend my time on this planet. I love that they’re trying to build repairable/upgradeable devices, but that goal doesn’t matter so much if people end up ditching your products for another option because they’re just tired of trying to fix it.
I’ll chime in with the opposite experience - I’ve owned an AMD Framework 13 since it came out, and had no durability issues with it whatsoever, and it’s been one of my 2 favorite computers I’ve ever owned. I’ve done one main board replacement that saved my butt after a bottle of gin fell over on top of it in transport.
Development and light gaming (on Linux, I very much appreciate their Linux support) have been great, and the reparability both gives me peace of mind, an upgrade path, and has already saved me quite a bit of money.
I’ve owned a framework since Batch 1. Durability has not been a problem for me. My original screen has a small chip in it from when I put it in a bag with something that had sharp edges and pressured the screen for a whole flight. Slowly growing. Otherwise, it’s been solid.
Same. I have a batch 1. There are quirks, which I expected and knew I am supporting a startup with little experience. I since have upgraded and put my old board into a cooler master case. This is so amazing, and what I cared about. I am still super happy with having bought the Framework and particular for tinkerers and people who will have a use for their old mainboards it’s amazing.
I get harbouring resentment for a company you felt sold then a bad product. But at the same time, you bought a laptop from a very inexperienced company which was brand new at making laptops, a pretty difficult product category to get right when you’re not just re-branding someone else’s white-label hardware.
3 years have passed since then, if I were in the market for a category which Framework competes in these days I would be inclined to look at more recent reviews and customer testimonials. I don’t think flaws in that 3 year old hardware is that relevant anymore. Not because 3 years is a particularly long time in the computer hardware business, but because it’s a really long time relative to the short life of this particular company.
I would agree that 3 years is enough time for a company to use their production lessons to improve their product. But nothing has changed in the Framework 13.
I don’t resent Framework. I think that’s putting words in my mouth. I just cannot, in good faith, recommend their products to people who need even a semi-durable machine. That’s just fact.
a very inexperienced company which was brand new at making laptops
Founded by people who had experience designing laptops already, and manufactured by a company that manufactures many laptops. Poor explanations for the problems, IMO.
I’ve had a 12th gen Intel since Sept 2022 (running NixOS btw) and I have not had any issues, I will admit it sits in one place 99% of the time. I might order the replacement hinge since mine is a bit floppy but not too big a deal.
As for the event, I was hoping for a minipc using the 395 and I got my wish. Bit pricey and not small enough for where I want to put it and I have no plans for AI work so it’s probably not the right machine for me.
I was originally interested in the HP machine coming with the same CPU (which should be small enough to fit) but I’ve been pricing an AMD 9950 and it comes out cheaper. I was also disappointed there wasn’t a sku with 385 Max w/64GB of RAM , which I might have have ordered to keep the cost down.
For reference a new machine is intended to replace a 10 year old Devils Canyon system.
I’ve also had my Framework 13 since beginning of 2022. I’ve had to do a hinge replacement, input cover replacement, and mainboard replacement. But I sort of expected that since it’s a young company and hardware is hard. And through all of it support was very responsive and helpful.
I would expect that nowadays the laptops are probably more solidly built than those early batches!
Support was definitely helpful. I just don’t have time or money to replace parts on my machine anymore.
From what I understand, the laptops aren’t any stronger. Even the Framework 16 just got some aftermarket/post-launch foam pads to put below the keyboard to alleviate the strain on the keyboard. The entire keyboard deck would flex.
The fact that these products have these flaws makes me wonder how Framework organizes its engineering priorities.
When compared to other similar laptops from brands like HP or Lenovo, how does the deck flex compare? I definitely feel sympathetic to not being better or on par with Apple - given the heaps of money Apple has for economies of scale + lots of mechanical engineers, but it would be a bit rough if mid-tier laptops in that category were far superior.
The deck flex is on par with or worse than an HP EliteBook circa 2019. The problem is that it’s incredibly easy to bend the entire frame of the machine, to the point where it interferes with the touchpad’s ability to click.
It’s really bad, bordering on unexcusable. The fact that there’s no concrete reinforcment says that they sacrificed build quality for repairability, which is equivalent to making a leaky boat with a very fast bilge pump.
I’m not sure what you’re doing to your laptop; how are you bending the entire frame of the machine?
It’s a new company that is largely doing right by open source, and especially open hardware. The quality isn’t incredible but it is worth its value, and I find these claims you’re making dubious.
It’s a fairly common flex point for the chassis, and a common support problem. The base of the mousepad, towards the front of the laptop where there’s a depression in the case, is where the majority of the flex is.
My laptop has seen nothing but daily, regular use. You can find the claims dubious, but others are having them too.
I’ll chime in too: I’ve had the Framework 13 AMD since it came out (mid 2023) and it has been great.
I upgraded the display after the new 2.8K panel came out, it took 2 minutes. Couple months later it developed some dead pixels, so they sent me a replacement. In the process of swapping it out, I accidentally tore the display cable. It took me a while to notice/debug it, but in the end it was just a $15 cable replacement that I’m fairly sure would have otherwise resulted in a full mainboard replacement for any other laptop. (When I had Macbooks, I lost count how many times Apple replaced the mainboard for the smallest thing.)
I haven’t been too precious with it, I toss it around like I did my Thinkpad before this. There’s some scuffs but it has been fine, perhaps the newer models are more sturdy? It’s comforting to know that if anything breaks, I’ll be able to fix it.
I also run NixOS on it, it does everything I need it to do, the battery life is great (8-10 hours of moderate use) and I’ll happily swap out the battery in a few more years once it starts losing capacity.
I spend so much of my life at the computer that feeling a sense of ownership over the components makes a lot of sense to me. I don’t want to feel like I’m living in a hotel.
It is, in fact, how I want to spend my time on this planet.
To add to the chorus, I bought a 12th gen intel framework 13 on release and it’s been flawless so far. Nixos worked out of the box. I love the 3:2 screen. I can totally believe that a small/young manufacturing company has quality control issues and some people are getting lemons, but the design itself seems solid to me.
On my old dell laptop I snapped all the usb ports on one side (by lifting up the other side while keyboard/mouse were still connected). Since they’re connected directly to the motherboard they weren’t repairable without buying a new cpu. If I did the same on the framework it would only break the $12 expansion cards and I wouldn’t even have to turn it off to replace them.
Later I dropped that same dell about 20cm on to a couch with the screen open. The impact swung the screen open all the way and snapped the hinges. They wanted me to send it back for repairs but I couldn’t handle the downtime, so for a year I just had the hinges duck-taped together. I’ve dropped my framework the same way, but because the screen opens the full 180 degrees it doesn’t leverage the hinges at all. And if it did break I’d be able to ship the part and replace it myself.
Not that I support the desktop offering as anything but waste, but the soldered RAM is apparently all about throughput:
We spent months working with AMD to explore ways around this but ultimately determined that it wasn’t technically feasible to land modular memory at high throughput with the 256-bit memory bus. (source)
I wish I could name something in good faith that was comparable to a hyper-repairable x86-64 laptop. Lenovo is pivoting towards repairability with the T14 Gen 5, but I can’t recommend that either yet.
Star Labs, System76, some old Thinkpad models.. there are “competitive” things, but few things that pitch the things Framework does.
While I agree on some of that, I must stress that I’ve had hardware that was fine until just one thing suddenly broke and everything was unusable. I’ll try an analogy: with repairability, if all your components are 99% reliable and working, the whole machine is at 99% but without it, even if all of them are at 99.9% instead, when you have 10 components, you’re not in a better situation overall.
And I say that while I need to finish going through support for a mainboard replacement due to fried USB ports on a first-gen machine (although not an initial batch). BTW, funnily I’m wondering if there’s an interaction with my yubikey. I also wish the chassis was a bit sturdier but that’s more of a wish.
As for thermals, while I think they could probably be better, the 11th gen Intel CPU that you have (just like I do) isn’t great at all: 13th gen ones are much better AFAIK.
I’ve experienced a full main board failure which led to me upgrading to a 12th gen on my own dime.
The thermal problems are still there, and their fans have some surprising QA problems that are exacerbated by thermal issues.
I wish I could excuse the fact that my machine feels like it’s going to explode even with power management. The fans grind after three replacements now, and I lack the energy and motivation to do a fourth.
I think 12th gen is pretty similar to 11th gen. I contemplated the upgrade for similar reasons but held off because I didn’t need to know and the gains seemed low. IIRC it’s really with 13th gen that Intel improved the CPUs. But I agree the thermals/power seems sub-par; I feel like it could definitely be better.
BTW, I just “remembered” that I use mine mostly on my desk and it’s not directly sitting on it which greatly improves its cooling (I can’t give hard numbers but I see the temps under load are better and max CPU frequency can be maintained).
Sorry to hear about the trouble with your Framework 13. To offer another data point: I have a 12th gen Framework 13 and haven’t needed to repair a thing, I’m still super happy with it. The frame-bending I’ve also not seen, it’s a super sturdy device for me.
The promising part of this paper is that they claim to both increase the robustness of planning (reduce the possibility of picking a really bad plan) AND often still increase performance in the case that the planner picks the best plan. Other robustness approaches that I’ve looked at often penalize the best/average case.
This article’s discussion of Zig’s error types reminds me of SML’s exception type.
In SML, user-defined types are closed sum types, there’s no subtyping. However, there’s a special case for exceptions, which are a special built-in type that is the only open sum type.
By the time I learned SML in the mid-1990s this special case was acknowledged as an expedient 1980s hack. There was more sophisticated type theory that could make open types a generally useful feature rather than a special case. (Very OO back then, but things have moved on further.)
Kind of funny to see an echo of that old mistake 30 years later.
While it is a mistake from type-theoretical point of view, from the ABI point of view it is not. It’s the ABI view that Zig takes — you can’t really make anything more efficient than an error-code, ABI-wise. So Zig starts with the assumption that, physically, an error is just an error code, and then adds type-system to make that as convenient to use as possible. Exceptions in Zig are not at all sum types, they don’t carry any data, unlike SML’s ones
This is in contrast to a more FP approach, where first you come up with a certain type system, and than the machine code to realize it is an implementation detail.
(note: zig errors are not sum types, they intentionally don’t carry data, so I don’t think comparison with SML exceptions or OCaml polymorphic variant is particularly illuminating)
zig errors are not sum types, they intentionally don’t carry data, so I don’t think comparison with SML exceptions or OCaml polymorphic variant is particularly illuminating
This feels a little ahistorical. The tracking issue for how to make zig errors carry data was open for 5 years and accrued ~100 comments before being decided against. IIRC the main concern was that easy error propagation would lead to people propagating data past the point where it should have been freed. There were also concerns about how inference would work and how conflicting tags would be handled, which is where the comparison to polymorphic variants, which have nearly identical challenges, is illuminating.
Whenever I do need to propagate data alongside errors, I end up with this kind of code:
This is frustrating because the whole selling point of zig errors is that you get precise inference and avoid the one-big-error-enum failure mode that rust tends towards (https://sled.rs/errors.html#why-does-this-matter). But as soon as I need to know anything about my errors, I have to make a sum type to carry the data and I end up with exactly the same problems I faced in rust, plus a bunch more boilerplate and potential for mistakes (eg the compiler won’t guarantee that whenever the above code returns error.foo then error_data == .foo).
I just spent way longer than I should have to debugging an issue of my project’s build not working on Windows given that all I had to work with from the zig compiler was an error: AccessDenied and the build command that failed. […] I think the fact that even the compiler can’t consistently implement this pattern points to it perhaps being too manual/tedious/unergonomic/difficult to expect the Zig ecosystem at large to do the same.
I remember one of the arguments for zigs error handling being that it encouraged good error handling by making it the path of least resistance. But the path of least resistance at the moment is definitely to not report any information about the error, and that shows up in many places in the stdlib.
Tangentially, I also find it incredibly frustrating when the spoken history edits from “X is desirable but we can’t figure out a way to make it work within the constraints” to “X was never desirable anyway, you’re better off without it, we intentionally don’t allow X”. There’s nothing wrong with butting up against constraints and I hate that the demands of marketing end up polluting our language design knowledge base.
Thanks for the extra context! Mea culpa, my phrasing was ambiguous and doesn’t communicate exactly what I intended. I didn’t intend to make a comment on the history of the thing, but rather explain the perspective that makes the current state self-consistent.
Basically, if you think about Zig errors as weird sum-types, you’ll get a wrong mental and will try to make them do things that they can’t do. Instead, the much more useful mental model is “better typed integer error codes”.
I do claim that Zig errors are not a half-way towards polymorphic variants, but rather fully complete statically types integers.
I am also definitely not claiming that “fancy codes” are adequate for error-management related tasks. I actually don’t know how to manage errors in general. Like, Rust has precisely the same problem where, by default, you get AccessDenied without any indication of what path was being accessed. My personal strategy here is:
make sure that the bulk of code doesn’t have to do any error handing whatsoever, that the domain is error free.
model errors in the domain. Eg, a compilation error is not actually an error, in the Result<E, T> sense. A compilation error is the same kind of outcome as normal compilation. The value of compilation task is (Vec<Diagnostics>, T).
if you can’t avoid real error-errors, than don’t try to be polymorphic in error reporting, and report errors at the point they originate, in the final form. Eg, if you are a web-server, don’t try to bubble the error to some sort of middle where, format an 404 html error page right on the spot where the error occurrs.
the much more useful mental model is “better typed integer error codes”
not claiming that “fancy codes” are adequate for error-management related tasks
Ah, I see where you’re coming from now. That makes more sense than what I initially interpreted.
model errors in the domain
don’t try to be polymorphic in error reporting
It’s interesting that if you follow those strategies then it kinda doesn’t make sense to have errorsets at all? If all the relevant information is outside the error code then why have an error code? Maybe I should just use return error.error everywhere.
This is the heart of my frustration with zig error handling. It’s almost a substantial improvement on everything else I’ve used, but then there is one missing piece and all the advice for dealing with that kinda sounds like “build your own error handling system and barely use zig’s error codes”.
I do use both of your strategies, but in both cases I find they really want to be anti-modular. Eg I end up with one big Diagnostic union and then if you only want to run the parser and not the whole compiler, the type signature still tells you that it might report Diagnostic.linker_symbol_not_found so you lose the value of the compiler checking that your error handling is exhaustive.
And when I look at other peoples zig code almost everything only uses zigs error codes and doesn’t bother with the rest of the information. Because error codes are very convenient and blessed and as soon as you try to do more than just an error code there is a big ramp in painfulness.
I really do feel like what I want is zigs error system but with polymorphic variants. I’ll just be good and not return data that needs to be freed, I promise :)
I really do feel like what I want is zigs error system but with polymorphic variants.
Can you do it yourself? I am imagining API like this:
pub fn caller(errors: *Errors) void {
helper(errors) catch |err| switch (err) {
error.FileNotFound => {
const path = errors.payload(error.FileNotFound, []const u8);
},
};
}
pub fn helper(errors: *Errors) error{FileNotFound}!void {
return errors.throw(error.FileNotFound, "my-file.zig");
}
const Errors = struct {
var error_stack = []u8;
pub fn throw(self: *@This(), comptime err: anytype, comptime payload_value: anytype) @TypeOf(err) {
comptime register(err, @TypeOf(payload_value));
// push palyload_value to error_stack;
}
pub fn payload(self: *@This(), comptime err: anytype, comptime PayloadType: type) PayloadType {
comptime register(err, PayloadType);
// pop @sizeOf(PayloadType) bytes off the stack
}
};
fn register(comptime err: anytype, comptime PayloadTyper: u32) void {
// Side-effect somehow to check for consistency:
// - @export a symbol to a custom section
// - or emit a `test` that does something at runtime during test
// - or @compileLog under a flag and parse compiler output.
}
Basically:
thread your own type-erased stack for error payloads
when throwing, push value’s bytes to the stack
when catching, pop bytes of the stack
to check consistency of types between throwing and catching, do some side-effecting comptime shenanigans. Now, zig doesn’t officially has side-effecting comptime, but, unofficially, @export, test, and @compileLog are side-effecting, so presumably you can cook something horrible with them? Can we just port https://github.com/dtolnay/linkme to zig?
Hmm, I feel the nerdsnipe, but that does seem like the rest of the owl :D
fn maybe_do_stuff() !void {
do_stuff() catch |err| {
switch (err) {
error.FileNotFound => {
return; // oops, should have dropped the payload, now later payloads might be UB, how do we check at comptime for code that isn't there?
},
else => |err| {
print(errors.payload(error.FileExploded, []const u8)); // how do we get the inferred type of err to check if this is a legit payload?
},
};
}
So the contract is that if it returns error.Foo then foo_error is defined and vice versa. It’s a sum type! Just one that the compiler won’t notice when I mess it up.
(I mean, obviously the pragmatic solution is to make my own language :D)
As Ralf responded to that post, it takes those optimizations as a given because that is how existing compilers work, and the “obvious” alternative is far away from the one we live in today.
Discussions of pointer provenance always seem to have someone show up to make this argument that the whole system is just wrong and should be remade from scratch. But the people who make this argument never seem to have anything to say about the reasons compilers and language specs for C-like languages are this way, or how their preferred approach might tackle those problems.
It would be much more interesting to discuss what this alternative approach would actually look like. Presumably very few people would be satisfied with merely falling back to LLVM or GCC’s -O0 output- so which optimizations should compilers perform, and how can they be justified?
Nowhere does it discuss the obvious alternative: those “seemingly” correct optimizations are not, in fact, correct.
The entire post is a discussion of which of the three optimizations should be deemed incorrect.
I think that the issue should be resolved by saying that pointers have provenance but integers do not, which means that it is the second optimization that is wrong … But ultimately, it will be up to the LLVM community to make this decision. All I can say for sure is that they have to make a choice, because the status quo of doing all three of these optimizations leads to incorrect compilation results.
Do you feel the same way about the strict aliasing rules? My understanding is that those were added back when C was competing for performance with Fortran. The performance penalty for “just doing what the programmer wrote” has only gotten larger since then.
I think the case for this with regards strict aliasing is actually stronger. That is, we know which optimizations need to assume strict aliasing and how much performance they win you - not a terrible amount, it turns out. That’s why GCC and Clang have -fno-strict-aliasing and why Linux uses it but is still plenty fast.
Provenance is much more central to how modern compilers work, its assumptions are so baked in that you can’t even tell these compilers “-fno-provenance” and see how much speed you lose.
That is, we have usable production compilers that don’t need strict aliasing, we don’t have any that don’t need provenance. So saying “strict aliasing is wrong” is a tenable position, and indeed Rust doesn’t have strict aliasing.
I spoke to Apple folks when their compiler team switched the default to strict aliasing. They reported that it made key workloads 5-10% faster and the fixes were much easier to do and upstream than I would have expected. My view of -fstrict-aliasing at the time was that it was a flag that let you generate incorrect code that ran slightly faster. They had actual data that convinced me otherwise.
In support of your argument, we do have wasm which has no UB. In some half-assed experiments I’ve found that no amount of optimizing wasm backend will make up for generating wasm with llvm O0, which seems like weak evidence that something in the c language semantics was necessary for better-than-O0 performance.
Poking through the generated code a lot of it is pointless stack loads/stores, which are impossible to eliminate unless you are allowed to make some assumptions about which pointers might alias local variables.
As a curmudgeonly old C guy, I hear you, but on the other hand, if I were to try to write that “everything else” down as a precise specification, I bet I couldn’t do it.
It feels like C is trying to be too many things at once. Maybe there should be a #pragma do_what_I_said for code that needs to relate to the actual hardware. I mean, when I was doing low-level C seriously, even that optimization of hoisting i+j out of the loop would have been annoying. If I wanted it hoisted, I woud have hoisted it myself!
Last time I thought about this, I concluded that without provenance, a huge proportion of optimizations would be invalid, so to me it seems like much more than just a nice to have.
I personally find attempts to create new SSA-based backends to be a little misguided, especially if you’re not approaching it from a theoretically strong and novel research based approach. LLVM is slow and complicated almost as a necessity, and if you want a faster backend you have to cull a large amount of optimizations. You might as well use something like Cranelift at that point. Of course, these things are not binary and a I wish the authors the best of luck in creating something cool and useful.
A small nit-pick from the blog:
Other important caveats are that LLVM doesn’t consider IR generation to be backend here
I think you’ll find this is the distinction between a compiler frontend and backend. The frontend produces the IR, which is optimized and turned into machine code (or assembly, object files, whatever) by the backend
Assuming it’s the same Yasser I know who’s writing an LLVM replacement, he doesn’t just have a theoretically strong approach, he has lots of input from “the guy” himself (Cliff Click, author of the Sea of Nodes paper in the 1990s, and the author of the SSA-based Hotspot Java Virtual Machine that billions of people rely on every day).
Yasser is young, and knows no limitations to his own abilities. In other words, he’s like most of us are – or were! – and while his success is not guaranteed, it’s people like him who know no limits that eventually will change the world. Hopefully for the better.
LLVM is slow and complicated almost as a necessity
No, I don’t believe this to be true. LLVM is slow and complicated mainly because it was a early learning experience for its young author. And instead of incorporating the things that were already known, and the things learned in the writing of it, it simply accepted those inefficiencies as “unavoidable” and didn’t bother to correct any of them. Inertia is a great ally, and also a horrible foe.
Assuming it’s the same Yasser I know who’s writing an LLVM replacement, he doesn’t just have a theoretically strong approach, he has lots of input from “the guy” himself (Cliff Click, author of the Sea of Nodes paper in the 1990s, and the author of the SSA-based Hotspot Java Virtual Machine that billions of people rely on every day).
Yasser is young, and knows no limitations to his own abilities. In other words, he’s like most of us are – or were! – and while his success is not guaranteed, it’s people like him who know no limits that eventually will change the world. Hopefully for the better.
That’s not really what I mean, I’m sure that Yasser is very talented. My contention is that creating a faster LLVM with comparable output is a research problem, and research problems should be approached scientifically. What is the hypothesis of this project? “A SSA backend based on sea of nodes can produce comparable output and significantly faster speeds”? This would be a significant advancement to Combining Analysis, Combining Optimizations, but as of right now there’s very little evidence to point to this being the case. I prefer smaller results that show promising signs, that can be used to build more significant results. Contrast this with the development of Copy and Patch by Haoran Xu, which is being developed into a remake of lua jit with significantly interesting results. I find his approach much more rigorous and also much more narrowly scoped and therefore much more interesting to follow.
As an aside, I think that some “pop” programmers have popularized the idea that all code is bloated and slow and needs to be completely re-written, and a lot of people have taken it on themselves to re-write huge portions of the stack themselves. And I think that’s great! As a fun personal project, but not the basis of a big serious project that I need to consider using. At least, I remain pretty skeptical of them early on.
No, I don’t believe this to be true. LLVM is slow and complicated mainly because it was a early learning experience for its young author. And instead of incorporating the things that were already known, and the things learned in the writing of it, it simply accepted those inefficiencies as “unavoidable” and didn’t bother to correct any of them. Inertia is a great ally, and also a horrible foe.
I find this to be a miss-characterization of the development of LLVM, Chris Lattner was 25 but he was also a PhD student, and he wasn’t the only author - Vikram Adve, his doctoral advisor was 37. Yes, it was a learning experience for Chris, because it was his PhD thesis, so it was still quite a significant advancement over the state of the art.
Also, LLVM developed inertia because it had a license that made it appealing for Apple to fund the development of. I think it will be quite hard for any new project to achieve half of the features of LLVM, and you might as well therefore attempt to modify LLVM if you have ideas on how to improve it.
My contention is that creating a faster LLVM with comparable output is a research problem, and research problems should be approached scientifically.
It’s not a research problem, it’s an engineering problem. He’s building something practical and useful to many, not exploring a novel concept and explaining it to a niche audience.
I find this to be a miss-characterization of the development of LLVM, Chris Lattner was 25 but he was also a PhD student, and he wasn’t the only author - Vikram Adve, his doctoral advisor was 37.
PhD students are notoriously terrible programmers, advisors even more so. They’re known for writing sloppy, hacky code that barely functions enough to get their thesis done. They embrace paradigms like Object-Oriented Programming which are popular among academics living in a bubble, but have been long outdated in the software engineering industry.
It’s not a research problem, it’s an engineering problem. He’s building something practical and useful to many, not exploring a novel concept and explaining it to a niche audience.
Writing an SSA backend that is faster and produces comparable output to LLVM is absolutely a research problem, and research does not have to be “exploring a novel concept and explaining it to a niche audience”.
Additionally, I will argue that all sufficiently sized engineering problems are research problems. You can’t just engineer your way through sufficiently large problems without research.
Again, I hope it does become practical and useful to many. But I think the foundations of the project make it unlikely to succeed in this front. I hope I’m wrong.
PhD students are notoriously terrible programmers, advisors even more so. They’re known for writing sloppy, hacky code that barely functions enough to get their thesis done. They embrace paradigms like Object-Oriented Programming which are popular among academics living in a bubble, but have been long outdated in the software engineering industry.
To me this feels just like something people say with no real evidence to back it up or disprove it. At best the evidence is anecdotal. I’ve met plenty of Drs of CS who write pretty good code, so I’m not sure why being a PhD would make you a poor coder besides other external factors like the stress of meeting the deadline for your thesis.
I’m not really trying to say anything about Chris Lattner, I’m just responding to your implied premise that being a PhD student means being a better programmer:
Chris Lattner was 25 but he was also a PhD student
This is the initial claim being made and therefore the one that has the burden of proof. My comment only serves to point out that this claim is not a valid premise because it is not widely accepted as fact.
It feels like saying “well Linus was 22 when he created Linux, and 22 year olds are notoriously terrible programmers”. Yes, there’s plenty to criticize from today’s POV, but that also just means the field has progressed, which is good.
Sure, the average Ph.D. student or their advisor is not a good programmer, but we’re talking about the people who created LLVM
Which Julia, Rust, Zig, Swift, and Odin all built on … It enabled a lot of open source to exist, which wouldn’t otherwise exist
If they were such horrible programmers, then surely it wouldn’t be a big deal to just ignore them and build something better. Why even build on top of something from these programmers?
Yet it will probably take a decade to build a good alternative (considering the targets, code gen speed, and runtime speed). And that will no doubt be done with lots of knowledge created by the LLVM project in the first place
LLVM also encouraged GCC to be more modular. AFAIK there were virtually no projects built on top of GCC – everything significant had to be maintained upstream in GCC because there were no public APIs
I’m not really trying to say anything about Chris Lattner, I’m just responding to [the] implied premise that being a PhD student means being a better programmer
We probably agree on a lot more of this than we disagree on. Any compiler back-end is a significant project of multiple man-years to do a half decent basic job of, let alone something that could replace LLVM. LLVM may not be optimal, but it has one significant advantage over most projects: It is available for use. That said, LLVM does have design flaws that have been pointed out over the years by people who have successfully built back-ends (i.e. not me – I’ve only done compiler front ends), and I have to assume that at least some portion of those criticisms are valid. Even Chris has admitted to various things that he wished he had done differently, so this isn’t some sort of blame game. I’m sure when Chris was building LLVM, a number of gatekeeping people were rolling their eyes at the audacity, but he pulled it off – criticisms notwithstanding. And I’m just trying avoid becoming one of those gatekeepers.
I agree with you on all the things stated here. I don’t intend to be unduly harsh or dismissive, and I do by no means wish to gatekeep the effort of anyone to do cool stuff.
LLVM is slow and complicated almost as a necessity
https://home.cit.tum.de/~engelke/pubs/2403-cgo.pdf notes a lot of bottlenecks that are architectural rather than algorithmic. The ir encoding is memory-heavy, is expensive to build, and requires a lot of pointer-chasing. Extensibility at every stage is implemented by virtual function calls per ir node, which are expensive even if you don’t override them. The linker first builds a complete elf object and then immediately parses it again. Etc.
A new codebase that does the exact same optimizations but is architected for current hardware could potentially be a lot faster. Memory density and access patterns matter a lot more now than they did 20 years ago. Same for multi-threading.
Is there something particular about SSA, that you find misguided, or do you mean that not being novel would have a hard time competing with an already developed, industry-standard implementation?
What other alternatives are there? I know that GraalVM uses a graph-based IR, for example, which I found very interesting. Do you (or someone else) have perhaps some option on how these compare?
Nothing about SSA is misguided. You hit it on the head - without being novel or strongly researched based into LLVMs deficiencies (a good example of what this would look like is pointed to in another comment), you’re unlikely to compete with LLVM
Everything is just some sort of re-imagining of something else. The devil is in the details though.
Smalltalk mixed persistent data and code together in a way that ended up being hard to maintain over time (eg schema migrations). Eve made all the data into a relational database, and those we know how to maintain.
I think airtable.com are the ones who actually nailed this though. Having an actual use case definitely helps design stay pragmatic.
(Anecdotally, the migrations I’ve done in Smalltalk have been about the same level of complexity as the migrations in DBMSs I’ve done. One important difference is lack of transactionality in (most) Smalltalks, meaning you generally perform migrations in a “stop the world” style as e.g. code is upgraded, or the image is (re)started, as required.)
It’s a shame GemStone never got an open-source clone (or, if it did, I never saw it). Seaside on Gemstone was amazing for web development. Every request ran in a separate transactional snapshot of the Smalltalk VM. This eventually hit scalability issues but they came much later than most other approaches.
I know there’s a smiley there, but this is an incredibly mild claim.
A sizeable majority of the entire worlds digital infrastructure runs on relational databases, at a level of scale and reliability that surpasses anything we’ve ever done with a smalltalk vm and that would blow the minds of a time-traveling programmer from the 80s.
I’m not saying there are no improvements to be made - I would be out of the job if databases were already perfect. But to the extent that we know how to maintain anything at all, our accumulated experience and expertise is overwhelmingly with relational databases. And we have by comparison barely any institutional knowledge of or experience with maintaining image-based systems.
So, I don’t know if this is really relevant to the overall discussion, but, I’ve learned as part of “received wisdom” that code in databases is different from data in databases, and is problematic - especially things like stored procedures, IIRC.
Why, specifically? I don’t know that. I haven’t used them. I can guess that versioning could be an issue? but then why isn’t this handled by migrations? Or is it that code stored in a DB requires more (or just different?) forethought than devs are used to thinking about with migrations? Maybe migration frameworks generally make assumptions that just don’t work well with stored procedures? I just don’t know. I wish I did, because I’ve often wondered about it. Input welcome from anyone who might see this reply.
random potential issues with storing behavior in DB I can imagine:
locally switching branches requires different procedures, but migration systems don’t handle this intelligently
stored procedures means doing more work on the DB, which is so hard to scale (tho perhaps this could sometimes not be the case, used judiciously)
I have not written any production grade smalltalk so I am really not very familiar with this. In what way did smalltalk make maintaining schema migrations difficult over time?
Also, I skimmed through airtable.com and I have no idea what they are about :( so can you please tell me?
Mary Rose Cook gave an 8 minute talk at HYTRADBOI 2022 that shows off a lot of the features, and how it scales smoothly from spreadsheet-like to writing javascript.
DBMS migrations, Smalltalk migrations, and, say, Debian package upgrades are all roughly the same kind of thing. Similar considerations apply to each and each is about as tricky to get right as the others, in my experience.
In the sense that you no longer have code in files, but code in a richer structure (a database of sorts): yes. Smalltalk IDEs let you slice-and-dice the codebase to present it any way you want; they offer multiple simultaneous views into the code; they are live.
In almost every other sense: no. Smalltalk is not reactive; Smalltalk is not a logic or a relational language; Smalltalk has no events (though most system libraries have the observer pattern somewhat well-integrated).
There was some work from the VPRI STEPS project to make a reactive Smalltalk dialect 15 or so years ago. Eve looks a lot like the prototypes I saw of that. It’s a shame it doesn’t seem to have been updated since 2017, I’d like to see a real system that enables end-user programming like this.
Ahh so many good ideas in this lang. such a shame.
I think one of the big issues I see with things like this is just how different they are from everything else, and so you’re having to make things work on all fronts. Just building a properly reactive system is an endeavor, then add in what appears to be logic lang semantics, a whole editor, a ui library (that history widget), a debugger…
Hey that’s ok! Its great that you went out and tried something. I’m kinda in the opposite - I’ve had ideas like this, but end up learning what I can before actually going into the task, and its a lot to learn.
Tho I’m coming around to a fixed point to where what I want to make is fairly consistent and I’m about at the skill level I think I need. It takes a long time with self study.
Copy-and-patch reuses the individual operator implementations from an interpreter…
A basic copy-and-patch approach seems like it would fail at implementing the query example from earlier in the article:
SELECT * FROM movies WHERE language = ‘english’ AND country = ‘usa’
SELECT * FROM movies WHERE language = ‘swahili’ AND country = ‘spain’
It seems like you might need to compile a template assuming multiple types of operations (table scan vs index lookup) and then pick the right template at runtime. That would get more complex when you start including joins (hash vs nested vs merge join), of which you could have an arbitrary number of combinations. Maybe there is a tractable set of basic queries where you could pre-compile and default to a slower interpreter for more complex queries. However, that might lead to some performance cliffs.
wasm all the way down
The major downside is that this means taking eg V8 as a dependency and having little to no control over its runtime behaviour.
There is a database/web platform called Convex that is using v8 as it’s database engine. It’s explicitly not a SQL based engine and it’s using the Javascript parts of v8, not the WASM parts. I guess the developers have an ops background and really distrust databases/query optimizers so they decided to take a no SQL approach. One issue that came up was that they had to make promise execution more deterministic, which involved digging into the guts of V8. (I believe that they talked about it on this episode of software unscripted.) I imagine that there could be other issues like this if you decided to pull an engine out of the browser.
The copy-and-patch paper was reporting compilation times for tcp-h in the 100s of microseconds, so you would be able to just compile both query plans on the fly. You might not even bother having an interpreter at that point.
Hey, thanks for taking the time to reply! I enjoyed the post.
For some context, I’m a data engineer so I might be thinking of more complicated queries on average. Take the following query:
select *
from a
join b (user_id)
left join c (user_id)
left join d (user_id)
where user_id = ?
If we are talking about:
table scan vs index lookup
hash vs nested vs merge
That might produce 4 tables * 2 lookup modes * 3 joins * 3 join modes = 96 different compilations, right? If we assume 100-500 microseconds per compilation target or 9.6-48 ms for the whole set of combinations. Or do you think that there would be enough information ahead of time to pare down the possibilities?
No, I mean when the query arrives you plan it and compile that plan. If copy-and-patch is as fast as claimed, it would only be a fraction of the planning time. No need to compile things in advance.
Okay, so there would be some parsing, query plan generation, and then a single copy-and-patch instance would be compiled. That makes sense. Thanks for clarifying!
The next step in the front-end is to transform the above AST to a logical query plan, this transformation will often output an AST but using different nomenclature, SELECT becomes Project, From becomes Datasource or Subquery in the case of nested queries for example.
The above logical plan, will refer to columns by their indices; you can think of the execution step as consuming arrays that represent rows (classic OLTP) or batches of rows or columns (vectorized OLAP).
The logical plan is then taken in by the query planner, the query planner is the unit that will be responsible for using TableScan or IndexScan depending on table statistics (like in Postgres) or just heuristics (OLAP systems don’t usually have indexes).
The query planner will emit a physical plan, where the Scan operator is transformed to either TableScan or IndexScan and do the same for other operators (Join for example would be transformed to HashJoin or LoopJoin… it depends on the query engine here).
There are other optimization steps that occur on the physical plan, but I won’t mention them here; at this step the query is finally ready for compilation.
Query compilation, involves emitting code that will operate on an input format that is already known, for example if the query engine uses Arrow then we can imagine query compilation as emitting; I’ll use Java here.
static void query(schema Schema, row []Object) Optional<Object[]> {
var col1 = 'english';
var col2 = 'usa';
var val1 = row.get(1);
var val2 = row.get(2);
if (val1 != null && val1.equals(col1) && val2 != null && val2.equals(col2)) {
return Some(row);
}
return None;
}
The above code, in the case of Apache Spark or Calcite, will be emitted to a raw string and then we’d invoke an embedded compiler like Janino to compile it to bytecode, load it into the JVM and run it against the inputs.
In the case of copy-and-patch, then what would happen is you would have a stencil that matches a query plan like the above, then it becomes a question of pattern matching. The compiler would then match the query plan against its own cache of plans, then load the bytecode for the stencil that describes the above query and patch the strings in the bytecode replace indices in the bytecode constant pool with english and usa.
In the case of LLVM your IR would simply reference to raw literals that get patched with the above; for example Amazon Redshift does something similar where there is a cache of compiled query plans that get patched and run at runtime.
I find it interesting that the JVM based ecosystem is not referenced in the article because it has, in my opinion, more mature query compilers. To my knowledge all query compilers that are based on using the LLVM stack are closed source (SingleStore, HyPer, Umbra) and not that popular unlike Spark or Calcite.
The core idea of building such query compilers boils down to thinking in Futamura projects which can be described as building specialized interpreters using some knowledge about input data, this paper [1] delves into the subject and gives a very clear exposition. In [2] there is a very nice and high level exposition of how query engines work with some effort you can add query compilation as a last step for fun :).
I’m so glad to see this, I’ve been throwing around the idea for an “LLVM for relational databases” for a while now, partly due to the seemingly backwards workflow of pasting source code from one language into string-literals of another language and losing all semblance of type-safety/IDE integration without editor-specific plugins etc.
I love sqlc but it still has drawbacks, and my mental model for why we keep running into this discussion of “raw sql vs orms” is there’s an abstraction layer bump/mismatch. SQL is a high level language, very high level, application/business logic code is “lower” level and a lot of these tools that work with SQL (sqlc, your average orm, prisma, etc) are essentially going up the abstraction ladder then back down again. As in, you write some ORM code, it gets “compiled upwards” into SQL which is then “compiled downwards” into a query plan. And of course you usually want to control your SQL at some point to fine-tune/control/optimise it (because “declarative” is a lie) so you bin the ORM and now spend time wiring up the outputs from flat tables (even if you want 2-3 layers of data thanks to a handy join) and then spend more time unwrapping that flat list back into its structured form.
And that being said, I really like the end-user experience (ignoring optimisation) of tools like Prisma, Ent, etc. they let me build very fast and the mental model of my data fits perfectly first time. But as soon as I need to break out of that cosy bubble, time has to be spent doing a lot of manual work.
And on top of that my general thesis is that SQL is stuck in 1973 and there’s almost zero “competition” or flourishing ecosystem of new language design simply due to the highly coupled nature of “I need to speak to Postgres because that’s our database and will forever be our database” so you’re locked in to the weird non-spec-compliant Postgres flavour of SQL forever.
This didn’t happen with application languages partly because of things like LLVM, JVM, etc. Someone, a fresh, young, innovative and smart person can come along and write a brand new language with interesting and challenging ideas to be critiqued, experimented with and maybe eventually deployed without needing to worry too much about all the CPU architectures and system details. They can just target LLVM and bloom a new idea into the world to be shared and enjoyed by others.
I suppose what I want is a baseline platform, like LLVM, on which smart people can try new things that actually have a chance of being used on real projects. There are plenty of toy databases out there with new ideas, that paper from Google with pipe-like syntax for SQL was awesome! But I’ll probably never get to use that because basically every real-world product is going to be using an incumbent database and be locked in to their flavour of SQL. Yes, you can “transpile” some interesting new language to SQL but then you run in to the original problem of “I need control over the actual underlying SQL query” and it all breaks down because you’re going up the abstraction ladder rather than down.
An ill-formed thought I suppose but hopefully you catch my drift!
“I need to speak to Postgres because that’s our database and will forever be our database” so you’re locked in to the weird non-spec-compliant Postgres flavour of SQL forever.
What are the most common extensions that you need Postgres for? i.e. the things that won’t run on MySQL. Is it things like
the actual queries - I thought most of this was standardized in ANSI SQL, except for fairly exotic queries
date and string functions? JSON functions?
durability settings, transactions ?
Is it totally foolhardy to try to be portable across Postgres and MySQL? I mean I guess that is what an ORM gives you, but ORMs are leaky abstractions, and you will often need to drop down.
I know that Python’s sqlite binding is badly hobbled by this ill-conceived idea to be portable across databases. In particular I remember dealing with silly hacks where it would textually paste SQL code around your code for transactions, so that sqlite would match the semantics of other databases. But this is brittle and can introduce performance problems
Going back to the original comment, I think the LLVM analogy only goes so far.
Because LLVM is not a stable API – it’s NOT a VM (which it says on the home page!) It’s a C++ library which can change at any release.
I think the problem is that MySQL has its own VM, and sqlite has its own VM, etc. And then the SQL layer is the common thing on top of that.
So it’s hard to introduce another VM layer – internally it would make sqlite a lot more complex, with no real benefit to sqlite itself
It’s similar to the problem of “why don’t we export a stable AST for every programming language?” The text representation is actually the stable thing; the internals can vary wildly between releases
I do agree that LLVM had a transformative effect on the language ecosystem though, with C++ Julia Rust Swift all using it.
The postgres thing was just an example, but I’ve consistently run into this over my career, and it’s not really about portability of software (I’ve never once worked in a team that actually switched databases - mostly because unless you have really specific needs, most tech teams will just pick their favourite, which in my experience is often Postgres. I think the “ability to switch database” isn’t a great argument tbh because it’s akin to rewriting your entire product in a different language, a hard sell for any business) it’s more the transferability of knowledge. I consistently forget `, “ and ’ are semantically different, I forget that some DBs want table.column or namespace.table.column format, I forget that Postgres does conflict clauses differently to SQLite and MySQL (from what I remember, postgres requires you to explicitly specify each conflict column, SQLite lets you just say “on any conflict, do this”)
One more (arguably, bad) example is migrations: I run an open source project that users deploy, it supports the 3 main DBs of the world, purely because it uses Ent which (the way I have it set up) dynamically nudges the schema to where it needs to be when the app boots up. I’d love to move to a more resilient migration strategy where I store a historical log of files with the “alter table” stuff in it, but for that I’d need to maintain 3 separate folders of slightly different generated migrations for each DB due to all the minor syntax differences. A bit of a pain tbh, and something that keeps me on the dynamic approach (which works well 99% of the time and runs into impossible changes 1% of the time, which I must manually fix and let users know how to fix for new releases, NOT ideal!)
But tbh, devil’s advocate: you could all argue those are just a skill issue on my part and I should learn my tools better!
But yes, you’re right, the LLVM comparison is half the way there. All these databases implement their own virtual layers (CRDB does a key-value store with an engine that translates SQL into KV scans, SQLite has some bytecode language, not sure what postgres does)
I don’t think this utopian idea of mine would ever work with incumbent products (if I ever get around to building it) but I do think there’s a gap in the market for a “not the fastest cutting edge DB in the world, but one with a really really nice developer experience that’s perfect for simple CRUD apps”
Though to clarify, my idea is not to literally use LLVM itself, but to construct a sort of new paradigm where there’s an LLVM-for-relational-queries that sits in the middle of some interface (be it SQL or a type-safe codegen solution for your app code) and the actual data store. But then what you have is a relatively neat relational data store and a (ideally) stable IR layer upon which anyone can build a cool new language without needing to understand/optimise the horrors of a relational algebra implementation!
A pipe dream. But perhaps something I’ll get to in my free time one day!
OLTP databases aren’t there yet. Probably because the storage isn’t pure data but mixes in things like mvcc and indexes, so it’s harder to find common ground between databases.
Oh neat, I’ve not looked into datafusion this looks in the same ballpark for sure! I did try to get my head around substrait but just couldn’t figure it out. Thanks for sharing!
I do feel like this may be one of those situations where my best way to learn is to just build something myself and run into all these complexities on my own then refer back to the prior art!
I’ve been saying for a while that I would love to see a Warren Abstract Machine for SQL. Maybe it wouldn’t be the most powerful or performant SQL database, but having a platform for language development seems like it would be very helpful.
This is one of the unique powers of declarativenominal type systems, to create a zero-runtime-cost wrapping type. I miss this ability in TypeScript where two aliases to one type, or two interfaces with equivalent requirements, are automatically interchangeable.
It’s not really about type systems so much as control over layout. Eg julia is dynamically typed but struct Foo; i::Int64; end is zero-runtime-cost.
The problem with javascript is universal layout. You can make {i: 42} almost zero-overhead with object shapes, but there’s no way to store it inline in an array or another object, so you’ll always incur the extra heap allocation.
Whereas in Julia you can make an Arrray{Foo} and it will have exactly the same layout as Array{Int64}.
In the course of writing this I noticed I should have said “nominal type system” instead of declarative:
I want to make sure we’re talking about the same thing… I think I take your point that the type system is not what provides the zero runtime cost aspect, but I think nominal typing is a factor in providing the safety benefit in these types. Here is what I mean:
There’s no error on the last line, because a ProductID provides guarantees that satisfy all of PersonID’s requirements. But I want an error, because I don’t intend them to be substitutable. Structurally identical objects are evaluated the same by all types. To get this safety I would need to make unique structure somehow.
Whereas in Swift’s nominal typing, the final line will fail because a PersonID is not a ProductID. Same layout does not mean type checks treat them the same, and that is useful.
struct ProductID { let id: String }
struct PersonID { let id: String }
let product: ProductID = .init(id: "product-id")
let person: PersonID = product // helpful error
You might still worry about collisions between keys in different libraries. Clojure had a neat solution to this - the symbol ::foo expands to :current-namespace/foo which is not equal to :some-other-namespace/foo. So I might end up writing {::product-id "blah"} which expands to {:net.scattered-thoughts.cool-app/product-id "blah"}.
In the limit, there isn’t much difference between nominal and structural typing, except that nominal types hide their unique value and structural types have to make it explicit. But making the unique value explicit can make some things simpler, like clojures automatic serde (https://www.scattered-thoughts.net/writing/the-shape-of-data).
This comment is verging on a long “well actually”, but I do think that structural types are underrated because of this specific confusion.
I’ve been working on a gradually structurally typed language where this can all be zero runtime cost too, like julia. You’d write:
product-id-type = struct[product-id: string]
buy = (product /product-id-type) {
...
}
buy([product-id: "big-red-bbq"])
// this is a type-error
buy([person-id: "big-red-bob"])
And a value of type product-id-type is exactly the same size as a string. The key is stored in the type itself.
Yeah, Clojure’s namespacing of map keys instead of whole product types is a great idea. Thanks for all the detail! I don’t have much to add but I appreciate it. Also count me a fan of any language that leans on gradual typing and kebab case.
using mind-blowing techniques like using Uint8Arrays as bit vectors
I think this is reflective of a real culture difference. For someone who is used to systems languages, a dense bit vector is a standard tool, as unremarkable as using a hashmap. One of the reasons that optimization is so frustrating in javascript is that many other standard tools, like an array of structs, are just not possible to represent. It feels like working with one hand tied behind your back. When you’re used to being able to write simple code that produces efficient representations, it’s hard to explain to someone who isn’t used to that just how much harder their life is.
Another reason is that the jit is very hard to reason about in a composable way. Changes in one part of your code can bubble up through heuristics to cause another, seemingly unrelated, part of your code to become slower. Some abstraction method might be cheap enough to use, except on Tuesdays when it blows your budget. Or your optimization improves performance in v8 today but tomorrow the heuristics change. These kinds of problems definitely happen in c and rust too (eg), but much less often.
Even very simply things can be hairy. Eg I often want to use a pair of integers as the key to a map. In javascript I have to convert them to a string first, which means that every insert causes a separate heap allocation and every lookup has to do unpredictable loads against several strings. Something that I would expect to be fast becomes slow for annoying reasons that are nearly impossible to work around.
It’s definitely possible to write fast code in javascript by just turing-tarpitting everything into primitive types, but the bigger the project it gets the larger the programmer overhead becomes.
I definitely recognize the value of a familiar language though, and if you can get within a few X of the performance without too much tarpit then maybe that’s worth it. But I wouldn’t volunteer myself for it. Too frustrating.
That line also struck me in particular. Using a Uint8Array as a bitvector is definitely something I’ve thought of, if not even implemented, a few times in JS. As you said, JS seems to actively work against you when you’re trying to optimize your code in ways that would work in other languages, so it disincentivizes you from trying to reason about performance.
There’s a lot to be said about consistency in object representation and opimizations in “dynamic” languages. OCaml is renowned to be a fast language not because it does crazy whole-program optimizations (the compiler is very fast precisely because it does no such thing) or because its memory representation of objects is particularly memory efficient (it actually allocates a lot; even int64s are heap allocated!), but because it is straightforward and consistent, which allows you to reason about performance very easily when you need it.
The thing about this is that it doesn’t even feel like it’s “optimization” – as much as non-pessimization of how one would store such data. A single block of memory seems straightforward, with the more complicated way being to have a bunch of separate heap allocated objects that are individually managed. That’s what ends up feeling nice to me about the languages JS is being contrasted with here: I actually just write the basic implementation that is a word-for-word reflection of the design and get a baseline performance right off the bat that lets me go for a long time (usually indefinitely) before having to consider ‘optimizing’. The productivity boost offered alongside not giving that seems to me like it would have to be really really high to be worth the tradeoff, and personally I haven’t felt like JS delivers on that.
That’s true, but there’s a big tradeoff in flexibility there. If you want to store these objects in continuous memory now your language needs to know how big these objects are and may be in the future, which effectively means you need a type system of some sort. If you want to move parts of this objects out of it then you need to explicitly copy them and free the original object, so either you or your language needs to know where these objects are and who they belong to. If you want to stack-allocate them now your functions also need to know how big these objects are, and if you want the functions to be generic you’ll have to go through a monomorphization step, which means your compiler will be slower.
I’m not saying these properties are not valuable, but the complexity needs to lie somewhere. JS trades the ease of not having to keep track of all of this with a more complicated and “pessimized” memory model, while languages that let you have this level of control are superficially much harder to use.
Makes sense, but I’ve just found that what I pay in each of those scenarios comes up when each of those concretely happens and is often / usually a payment that seems consistent with the requirement and can be done in a way that’s pertinent to the context (I tend to not try to address all of those upfront before they actually happen, at this point). However, the performance requirement tends to either be always present or even if not present just be a mark of quality or discovery regardless (eg. if I can just have more entities in my game in my initial prototype I may discover kinds of gameplay I wouldn’t have otherwise).
I think I don’t consider those tradeoffs you’re listing very negatively because the type system usually comes with the language I use and I consider this to be a baseline for various sorts of productivity and implementation benefits at this point (which seems to also be demonstrated in the case of JS by how much TypeScript is a thing – it’s just that TypeScript makes you write types but doesn’t give you the performance and some other benefits that would’ve come with them elsewhere). Similar with stack allocation and value types.
I’m not seeing how the other side of that tradeoff is ‘flexibility’ specifically though, because my experience is that the type system lets me refactor to other approaches with help from the language’s tooling, and so far the discussion has been about how the JS approach is actually inflexible regarding that.
Re: ‘the complexity needs to lie somewhere’ – the point is that in the base case (eg. iterating an array of structs of numbers) there is actually less essential complexity and so it may not have to lie anywhere. It’s just that JS inserts the complexity even when it wasn’t necessary. It seems more prudent to only introduce the complexity when the circumstances that require it do arise, and to orient the design to those requirements contextually.
All that said, I’m mostly just sharing my current personal leaning on things that leads to my choice of tools. I just feel like I wasn’t actually that much more productive in JS-like environments, maybe even less so. On the whole I’m a supporter of trying various languages and tools, learning from them and using whatever one wants to.
I agree with everything you said. I’m just not sure that we should be defaulting to writing everything in Rust (as much as I like it) or some other similar language. I think I wish for some sort of middle ground.
Agreed on that. I definitely don’t think that the current languages – on any of the points in this spectrum – are clear winners yet, or that it’s a solved space. I avoided mentioning any particular languages on the ‘more control’ side for that reason.
Aside:
Lately I’ve been playing with C along with a custom code generator (which generates container types like TArray for each T and also ‘trait’ functions like TDealloc and TClone etc. that recurse on fields) of all things. Far from something I’d recommend seriously for sure, just a fun experiment for side projects. It’s been a surprising combination of ergonomics and clarity. Turns out __attribute__((cleanup)) in GCC and clang allow implementing defer as well. Would be cool to get any basic attempt at flow-sensitive analysis (like borrow-checking) working in this if possible, but I’ve been carrying on with heavy use of lsan (runs every 5 seconds throughout app lifetime) and asan during development for now.
This is an old prototype. I ended up making a language for it from scratch so that I could attach provenance metadata to values, making them directly editable even when far removed from their original source.
https://www.scattered-thoughts.net/log/0027#preimp
https://x.com/sc13ts/status/1564759255198351360/video/1
I never wrote up most of that work. I still like the ideas though.
Also if I had ever finished editing this I wouldn’t have buried the lede quite so much.
In Lil, a scripting language which includes the concepts of a database-style table and a query language for it, I might tackle tackle this problem something like the following:
But Lil isn’t just a scripting language; it’s also part of the Decker ecosystem, which has its own facilities for representing tables as manipulable and automatically serialized “grids”. A Decker-based user interface might contain a script something like this to compute the two output tables from the inputs:
But can you directly edit a row in the “untagged” view, or do you have to manually navigate back to the source data?
In the top example, i’m using Lil’s
insertquery form; a table literal much like your list-of-dicts in the Clojure version. It’s possible to hardcode tables like that in scripts within the Decker environment, but it’s more common for tables to be “stored” in grid widgets. The data within a grid can be directly edited:http://beyondloom.com/decker/tour.html#Grids
And modifying a grid manually can fire events which cause other “views” to be recomputed:
http://beyondloom.com/decker/tour.html#Scripting%203
I think this is still missing the point. If I’m looking at a row in the “untagged” view that you created, how do I add a tag to it? My goal was that you just directly edit the tag, and that flows back to the underlying data (demos in https://lobste.rs/s/jcsfbx/program_is_database_is_interface#c_i80teg). As opposed to having to navigate to the source table and scroll around for the row in question.
There are some previous research systems that do this using lenses, but I found that the lens laws are often violated by regular ui idioms and that just tracking per-value provenance was good enough most of the time.
Long post, many thoughts, but I don’t feel like doing a lot of editing, so apologies in front for unfiltered feedback! I don’t mean the tone I will use here :-)
The start of the article is 🤌, but it sort of does’t live to my expectations. I feel this is mostly about extensible, forward compatible enums, which is quite neat (I didn’t realize that “I want to add a field to all these enum variants” admits such an elegant solution), but I don’t think solves my problems with error handling facilities in languages.
Basically, this post feels like it attacks “make complicated things possible” part of the problem, and, sure, if you add across and along non-exhaustiveness,
!__for warnings, auto-delegation to turn an enum into struct, a capability system to track panics, you can solve everything.But the problem I see with error handling is that we don’t know how to make simple things easy. That’s especially true in Rust, of course, but it seems that in every language a simple way to go about error handling leads to some pretty significant drawbacks, and the money question is not how can we add extra knobs to handle all of the requirements, but whether there’s some simple idea that kinda solves stuff?
Like, for example, the sled problem — we want every function to be precise about its specific error conditions, but, in practice, the stable equilibrium is one error type for library. Everr (fabulous name by the way, great job) suggest
But this sucks! With a nominal type system, having to name every function, and every function’s error type is very much not easy, and even if you add a bunch of tooling support, the result would still not be easy.
Another simple-things-are-hard problem in error handling is exposing details. If you write a library A, and it uses a library B, and B is an implementation detail, then a common API design pitfall is to leak B through your error types (either directly, by including B variants, or indirectly, by allowing downcasing to B). The problem here isn’t that it’s impossible to either expose or hide B properly. There’s a bunch of techniques available for that (but I belive that Everr makes them nicer and more powerful). The problem is that you need to decide what do you do, and that is hard. You need pretty high level of discipline and experience to even note that this is a problem.
Or another common pitfall of type-based error types, where
is often a bug, because the actual picture is
That is, that the fact that you can aggregate errors based on types doesn’t mean that you should.
I have no idea how to handle errors in general! I just don’t have bullet proof recipes, every time it is “well, let’s look at your specific situation, shall we?”. Super annoying!
I don’t think that a lack of language mechanisms is my problem. What I lack is a gang-of-four book for patterns of error management (I mean, I have such a book at my head obviously, and I consult it often when writing code, but I can’t condense it to a single-paragraph to put into project’s style guide and call it a day).
Assorted smaller thoughts:
For an article about systems programming language, it is surprising that no space is dedicated to the ABI. How exactly do you raise in catch errors, in terms of which bytes go into which register, I feel is an unsolved problem. Returning values is allegedly slow. Unwinding is, counterintuitively, faster (see Duffy’s post & the recent talk on C++ exceptions in embedded (has anyone reproduced that result in particular?)). To avoid potential ambiguity: rust-style error handling, and Java-style exceptions differ on two orthogonal axis:
I am strictly speaking about the second one.
And than, there’s Zig, and than there’s this paper by Sutter of several years ago which says that “actually, you do want to return an integer” to be fast.
Heap exhaustion is not the central example of OutOfMemory error. The central example is someone passing you a malformed
gifimage whose declared size is 67PiB. That’s the sort of thing that you need to be robust to, a single rouge allocation due to a bug/malformed input.No these data here, but, anecdotally, eyeballing Zig the code that has both allocator parameter, try, and defer/errdefer usually tends to reveal errors.
The Rust allocator API is very much not what Zig is doing. https://ziglang.org/download/0.14.0/release-notes.html#Embracing-Unmanaged-Style-Containers is not at all that.
I used to think the what Rust does, with must_use, is the right thing, and was hesitant of swift approach of requiring everything to be used. After using Zig, I am sold though, no need to other think this, a non-void function whose result is unused and is not
_ =should be a compilation error. The amount of false positives is vanishingly small.https://mcyoung.xyz/2024/04/17/calling-convention/ had an interesting idea. Change the abi so that the error/success payloads of Result are passed as out parameters, and then just return the Ok/Err tag. That seems like it allows the best of both worlds - effectively automating the common pattern used in zig and making it type-safe.
Or, as @david_chisnall suggested, use the carry flag for ok/err https://lobste.rs/s/mzqv3t/why_i_prefer_exceptions_error_values#c_br9wzy
I think this is true in the common case where an error did not occur. Returning error information adds overhead to both the caller and callee, whereas catch/throw has the famous “zero overhead.” On the other hand, when an error does occur, unwinding the stack is significantly slower because a bunch of compiler-generated metadata has to be looked up and processed for each active stack frame.
The talk I watched (i don’t remember who gave it) was primarily about code size, not performance. The common wisdom being that using C++ exceptions bloats your code with all those compiler-generated tables annd extra code for running destructors during unwinding.
Hey Alex, thanks for taking the time to read and share your thoughts. I always appreciate reading your blog posts, so thank you for the specific feedback on this post.
It would be helpful to have an operational definition of “simple” here with one or two examples before I attempt to answer this. 😅
For example, if there is a guideline that by default, an error should not expose structure, and just expose an interface like:
and the implementation for this were to be derived using a macro (or comptime machinery in Zig), would that be considered “simple”?
Thanks for linking that blog post, I hadn’t read it earlier. This point stands out to me in particular:
This goes directly against the Rust conventions RFC, which recommends using panics for “catastrophic errors”. I’ve seen this similar tendency in Go codebases, where people will put every kind of error under
error, even if it’s technically a serious invariant violation (like a bounds check failure, which does trigger a panic!).Based on Duffy’s writing on Midori, it feels like a Midori programmer would probably be more likely to use “abandonment” (panic) than a Rust/Go programmer in this kind of situation, given the built-in Erlang-style fault-tolerant architecture.
Right, so with Everr’s type system, you could write your code as returning only
MyLibraryError, and then the language server can refactor the functions which need specific error conditions to instead returnMyLibraryError:union+[.Case1 | .Case2].This is a fair criticism. In that section, I originally intended to describe a system for regions/different sub-heaps based on some of the research on Verona (and in that context, “heap exhaustion” would mean “this sub-heap is exhausted”, not “the process heap is quite high for the running system”), but then I punted on that because I didn’t feel confident in Verona’s system, so I moved that description to the appendix.
I will update this.
I personally prefer Swift’s approach of warning instead of a hard error, given that iterating on code becomes more fiddly if you need to keep putting/removing
_(speaking from first-hand experience with Go).However, the point you’ve quoted here is talking about something slightly different. It’s saying that using the same shorthand for discarding ordinary and discarding errors is itself error-prone. Discarding errors should require noisier syntax (in lint form), because an error being discarded is likely to carry higher risk than a success value being discarded.
Perhaps a good idea for a IronBeetle episode? I’m slowly working my way through the list; maybe you’ve already covered this in one of them. 😄
I omitted this because:
Since this point is primarily about performance, it doesn’t make sense for me to speculate about designs without having concrete measurements. Performing any kind of realistic measurement would likely be a fair amount of work.
I didn’t realize it was an “unsolved problem” but rather my working assumption was that “for ABI, the small number of people working on it pretty much know what all their options are, so it doesn’t make sense for me to tell them that”. For example, if you only care about 64-bit machines, perhaps you’re fine with burning a register on errors specifically (like Swift). For larger errors, you could reuse the same register as an out-parameter (as described in mcyoung’s post linked by Jamie).
Good idea! I’ll do an error management episode this Thursday then!
Thanks for organizing this event! It was a good use of time – the talks were interesting and well paced
I’d personally be biased toward more “experience reports” than “new ideas”, but I think it’s a good format, and maybe others can take some cues for a variety of events in the same format
I wouldn’t have minded some more organized discussion afterward, although I don’t have any ideas on how to make that happen
I’m also glad Zulip worked out, since TBH if it was on Slack or Discord I may not have joined :)
Yeah, it’s a weird common knowledge problem. There could have been a big discussion the day afterwards if everyone believed there was going to be one. I have thought about spreading the talks out more to give the discussion more room to breathe. Maybe one block per evening or similar.
@jamii is that a typo??? How was Zulip so expensive?!
They add details further down:
Paying $1k to have rapid-response support from Zulip for a live conference sounds pretty reasonable.
Fwiw if I was running a non-profit conference I would have just done it on the free tier and it probably would have been fine. I just felt that if I’m charging money for tickets then I should also be paying for the platform.
I don’t understand how this is really any different than typical serialization.The recursion depth is only 1 deep, but that’s enough for you to still need to write the code that reflects all the fields and follows the pointers. He says you only need one writev call, but that would be true even with an unlimited number of pointers, the whole point of writev is that it can do an arbitrary amount of work. He says it makes it so the on disk format can be exactly the same, but then admits it actually can’t be because of the pointers for the arrays and hash tables, and if your serialization already has to be smart enough to handle that it’s only a tiny bit more work to handle to arbitrary depth. Also how do you do a hash table where the values are variable length arrays? If you try to express it as indexes you need a variable length array of variable length arrays, and then you can’t reduce that to arrays of indexes, somewhere you need an array of pointers. If in this scheme you’re allowed to nest hash tables and dynamic length arrays then you are long the problem of handling arbitrary depth and you’re back at square one. Am I missing something here?
Maybe useful context is that this is how the incremental parts of the zig compiler is being written. It only needs to write/read O(1) chunks of memory to save/load the entire state of the compiler, whereas if it used internal pointers it would be O(amount of code being compiled).
There’s a real example here https://github.com/ziglang/zig/blob/master/src/link/Wasm.zig#L50-L305
Zig hashmaps can have custom hash/comparison functions. So you can have one big array of data, a ‘slice’ type which is a offset+length into that array, and a hashmap that knows how to look up the data for those ‘slices’.
But they allow hashmaps and variable length arrays so it’s actually still O(n) ? Those all point to separate blocks of memory do they not?
The number of hashmaps and variable length arrays is constant, and does not increase as you compile more code
They’re not as soon as you have
HashMap<K, Vec<T>>orVec<Vec<T>>, no?HashMap<K, Vec> would become (HashMap<K, OffsetAndLen>, Vec) as I explained above, giving a constant 2 regions of memory. Just like the string examples in the talk.
That’s not equivalent, you can no longer resize one Vec independently of the others. That only works if their length changes rarely.
Alex put up a transcript at https://transactional.blog/talk/enough-with-all-the-raft.
Having owned a Framework since April of 2022, I cannot recommend them to people who need even basic durability in their devices. Since then, I have done two mainboard replacements, two top cover replacements, a hinge replacement, a battery replacement, several glue jobs after the captive screw hubs sheared from the plastic backing…
It’s just such an absurdly fragile device with incredibly poor thermals. They sacrificed a ton of desirable features to make the laptop repairable, but ultimately have released a set of devices that, when used in real-world settings, end with you repairing the device more often than not. And these repairs are often non-trivial.
I will personally be migrating to another machine. The Framework 12’s focus on durability may be trending in the right direction, but to regain trust, I’d need to see things like drop and wear tests. A laptop that can be repaired, but needs constant upkeep/incredibly delicate handling, is ultimately not an actual consumer device, but a hobbyist device.
Maybe they’ll get better in a few years. Maybe the Framework 12 will be better. Their new focus on AI, the soldered RAM in the desktop offering, and the failure to address the flimsy plastic chassis innards, among other things, mean that they have a long way to go.
It’s definitely a “be part of the community that helps solve our product problems” sort of feeling.
I have an AMD FW13, and was trying to figure out why it loses 50+% of its battery charge overnight when I close the lid, because I don’t use this computer every single day and don’t want to have remember to charge yet another device.
So I check the basics-I’m running their officially supported Linux distro, BIOS is current, etc. And half an hour into reading forum threads about diagnosing sleep power draw, I realize that this is not how I want to spend my time on this planet. I love that they’re trying to build repairable/upgradeable devices, but that goal doesn’t matter so much if people end up ditching your products for another option because they’re just tired of trying to fix it.
I’ll chime in with the opposite experience - I’ve owned an AMD Framework 13 since it came out, and had no durability issues with it whatsoever, and it’s been one of my 2 favorite computers I’ve ever owned. I’ve done one main board replacement that saved my butt after a bottle of gin fell over on top of it in transport.
Development and light gaming (on Linux, I very much appreciate their Linux support) have been great, and the reparability both gives me peace of mind, an upgrade path, and has already saved me quite a bit of money.
I’ve owned a framework since Batch 1. Durability has not been a problem for me. My original screen has a small chip in it from when I put it in a bag with something that had sharp edges and pressured the screen for a whole flight. Slowly growing. Otherwise, it’s been solid.
Same. I have a batch 1. There are quirks, which I expected and knew I am supporting a startup with little experience. I since have upgraded and put my old board into a cooler master case. This is so amazing, and what I cared about. I am still super happy with having bought the Framework and particular for tinkerers and people who will have a use for their old mainboards it’s amazing.
I get harbouring resentment for a company you felt sold then a bad product. But at the same time, you bought a laptop from a very inexperienced company which was brand new at making laptops, a pretty difficult product category to get right when you’re not just re-branding someone else’s white-label hardware.
3 years have passed since then, if I were in the market for a category which Framework competes in these days I would be inclined to look at more recent reviews and customer testimonials. I don’t think flaws in that 3 year old hardware is that relevant anymore. Not because 3 years is a particularly long time in the computer hardware business, but because it’s a really long time relative to the short life of this particular company.
I would agree that 3 years is enough time for a company to use their production lessons to improve their product. But nothing has changed in the Framework 13.
I don’t resent Framework. I think that’s putting words in my mouth. I just cannot, in good faith, recommend their products to people who need even a semi-durable machine. That’s just fact.
Founded by people who had experience designing laptops already, and manufactured by a company that manufactures many laptops. Poor explanations for the problems, IMO.
I’ve had a 12th gen Intel since Sept 2022 (running NixOS btw) and I have not had any issues, I will admit it sits in one place 99% of the time. I might order the replacement hinge since mine is a bit floppy but not too big a deal.
As for the event, I was hoping for a minipc using the 395 and I got my wish. Bit pricey and not small enough for where I want to put it and I have no plans for AI work so it’s probably not the right machine for me.
I was originally interested in the HP machine coming with the same CPU (which should be small enough to fit) but I’ve been pricing an AMD 9950 and it comes out cheaper. I was also disappointed there wasn’t a sku with 385 Max w/64GB of RAM , which I might have have ordered to keep the cost down.
For reference a new machine is intended to replace a 10 year old Devils Canyon system.
I’ve also had my Framework 13 since beginning of 2022. I’ve had to do a hinge replacement, input cover replacement, and mainboard replacement. But I sort of expected that since it’s a young company and hardware is hard. And through all of it support was very responsive and helpful.
I would expect that nowadays the laptops are probably more solidly built than those early batches!
Support was definitely helpful. I just don’t have time or money to replace parts on my machine anymore.
From what I understand, the laptops aren’t any stronger. Even the Framework 16 just got some aftermarket/post-launch foam pads to put below the keyboard to alleviate the strain on the keyboard. The entire keyboard deck would flex.
The fact that these products have these flaws makes me wonder how Framework organizes its engineering priorities.
When compared to other similar laptops from brands like HP or Lenovo, how does the deck flex compare? I definitely feel sympathetic to not being better or on par with Apple - given the heaps of money Apple has for economies of scale + lots of mechanical engineers, but it would be a bit rough if mid-tier laptops in that category were far superior.
The deck flex is on par with or worse than an HP EliteBook circa 2019. The problem is that it’s incredibly easy to bend the entire frame of the machine, to the point where it interferes with the touchpad’s ability to click.
It’s really bad, bordering on unexcusable. The fact that there’s no concrete reinforcment says that they sacrificed build quality for repairability, which is equivalent to making a leaky boat with a very fast bilge pump.
I’m not sure what you’re doing to your laptop; how are you bending the entire frame of the machine?
It’s a new company that is largely doing right by open source, and especially open hardware. The quality isn’t incredible but it is worth its value, and I find these claims you’re making dubious.
It’s a fairly common flex point for the chassis, and a common support problem. The base of the mousepad, towards the front of the laptop where there’s a depression in the case, is where the majority of the flex is.
My laptop has seen nothing but daily, regular use. You can find the claims dubious, but others are having them too.
This has been my experience with the Framework. It’s not Apple hardware, which is best in class all around, but it is on-par with my Dell XPS.
I’ll chime in too: I’ve had the Framework 13 AMD since it came out (mid 2023) and it has been great.
I upgraded the display after the new 2.8K panel came out, it took 2 minutes. Couple months later it developed some dead pixels, so they sent me a replacement. In the process of swapping it out, I accidentally tore the display cable. It took me a while to notice/debug it, but in the end it was just a $15 cable replacement that I’m fairly sure would have otherwise resulted in a full mainboard replacement for any other laptop. (When I had Macbooks, I lost count how many times Apple replaced the mainboard for the smallest thing.)
I haven’t been too precious with it, I toss it around like I did my Thinkpad before this. There’s some scuffs but it has been fine, perhaps the newer models are more sturdy? It’s comforting to know that if anything breaks, I’ll be able to fix it.
I also run NixOS on it, it does everything I need it to do, the battery life is great (8-10 hours of moderate use) and I’ll happily swap out the battery in a few more years once it starts losing capacity.
I spend so much of my life at the computer that feeling a sense of ownership over the components makes a lot of sense to me. I don’t want to feel like I’m living in a hotel.
It is, in fact, how I want to spend my time on this planet.
To add to the chorus, I bought a 12th gen intel framework 13 on release and it’s been flawless so far. Nixos worked out of the box. I love the 3:2 screen. I can totally believe that a small/young manufacturing company has quality control issues and some people are getting lemons, but the design itself seems solid to me.
On my old dell laptop I snapped all the usb ports on one side (by lifting up the other side while keyboard/mouse were still connected). Since they’re connected directly to the motherboard they weren’t repairable without buying a new cpu. If I did the same on the framework it would only break the $12 expansion cards and I wouldn’t even have to turn it off to replace them.
Later I dropped that same dell about 20cm on to a couch with the screen open. The impact swung the screen open all the way and snapped the hinges. They wanted me to send it back for repairs but I couldn’t handle the downtime, so for a year I just had the hinges duck-taped together. I’ve dropped my framework the same way, but because the screen opens the full 180 degrees it doesn’t leverage the hinges at all. And if it did break I’d be able to ship the part and replace it myself.
Not that I support the desktop offering as anything but waste, but the soldered RAM is apparently all about throughput:
With the focus of the desktop being “AI applications” that prioritize high throughpout, I’d say they could’ve gone with an entirely different chip.
I get the engineering constraint, but the reason for the constraint is something I disagree with.
Who else is making something competitive?
I wish I could name something in good faith that was comparable to a hyper-repairable x86-64 laptop. Lenovo is pivoting towards repairability with the T14 Gen 5, but I can’t recommend that either yet.
Star Labs, System76, some old Thinkpad models.. there are “competitive” things, but few things that pitch the things Framework does.
While I agree on some of that, I must stress that I’ve had hardware that was fine until just one thing suddenly broke and everything was unusable. I’ll try an analogy: with repairability, if all your components are 99% reliable and working, the whole machine is at 99% but without it, even if all of them are at 99.9% instead, when you have 10 components, you’re not in a better situation overall.
And I say that while I need to finish going through support for a mainboard replacement due to fried USB ports on a first-gen machine (although not an initial batch). BTW, funnily I’m wondering if there’s an interaction with my yubikey. I also wish the chassis was a bit sturdier but that’s more of a wish.
As for thermals, while I think they could probably be better, the 11th gen Intel CPU that you have (just like I do) isn’t great at all: 13th gen ones are much better AFAIK.
I’ve experienced a full main board failure which led to me upgrading to a 12th gen on my own dime.
The thermal problems are still there, and their fans have some surprising QA problems that are exacerbated by thermal issues.
I wish I could excuse the fact that my machine feels like it’s going to explode even with power management. The fans grind after three replacements now, and I lack the energy and motivation to do a fourth.
I think 12th gen is pretty similar to 11th gen. I contemplated the upgrade for similar reasons but held off because I didn’t need to know and the gains seemed low. IIRC it’s really with 13th gen that Intel improved the CPUs. But I agree the thermals/power seems sub-par; I feel like it could definitely be better.
BTW, I just “remembered” that I use mine mostly on my desk and it’s not directly sitting on it which greatly improves its cooling (I can’t give hard numbers but I see the temps under load are better and max CPU frequency can be maintained).
Sorry to hear about the trouble with your Framework 13. To offer another data point: I have a 12th gen Framework 13 and haven’t needed to repair a thing, I’m still super happy with it. The frame-bending I’ve also not seen, it’s a super sturdy device for me.
I can second that. I’ve had a 12th gen Intel system since late 2022 and no issues of the sort. Even dropping it once did nothing to it
The promising part of this paper is that they claim to both increase the robustness of planning (reduce the possibility of picking a really bad plan) AND often still increase performance in the case that the planner picks the best plan. Other robustness approaches that I’ve looked at often penalize the best/average case.
This article’s discussion of Zig’s error types reminds me of SML’s exception type.
In SML, user-defined types are closed sum types, there’s no subtyping. However, there’s a special case for exceptions, which are a special built-in type that is the only open sum type.
By the time I learned SML in the mid-1990s this special case was acknowledged as an expedient 1980s hack. There was more sophisticated type theory that could make open types a generally useful feature rather than a special case. (Very OO back then, but things have moved on further.)
Kind of funny to see an echo of that old mistake 30 years later.
While it is a mistake from type-theoretical point of view, from the ABI point of view it is not. It’s the ABI view that Zig takes — you can’t really make anything more efficient than an error-code, ABI-wise. So Zig starts with the assumption that, physically, an error is just an error code, and then adds type-system to make that as convenient to use as possible. Exceptions in Zig are not at all sum types, they don’t carry any data, unlike SML’s ones
This is in contrast to a more FP approach, where first you come up with a certain type system, and than the machine code to realize it is an implementation detail.
(note: zig errors are not sum types, they intentionally don’t carry data, so I don’t think comparison with SML exceptions or OCaml polymorphic variant is particularly illuminating)
This feels a little ahistorical. The tracking issue for how to make zig errors carry data was open for 5 years and accrued ~100 comments before being decided against. IIRC the main concern was that easy error propagation would lead to people propagating data past the point where it should have been freed. There were also concerns about how inference would work and how conflicting tags would be handled, which is where the comparison to polymorphic variants, which have nearly identical challenges, is illuminating.
Whenever I do need to propagate data alongside errors, I end up with this kind of code:
This is frustrating because the whole selling point of zig errors is that you get precise inference and avoid the one-big-error-enum failure mode that rust tends towards (https://sled.rs/errors.html#why-does-this-matter). But as soon as I need to know anything about my errors, I have to make a sum type to carry the data and I end up with exactly the same problems I faced in rust, plus a bunch more boilerplate and potential for mistakes (eg the compiler won’t guarantee that whenever the above code returns error.foo then error_data == .foo).
I think https://github.com/ziglang/zig/issues/2647#issuecomment-1444790576 is also worth considering:
I remember one of the arguments for zigs error handling being that it encouraged good error handling by making it the path of least resistance. But the path of least resistance at the moment is definitely to not report any information about the error, and that shows up in many places in the stdlib.
Tangentially, I also find it incredibly frustrating when the spoken history edits from “X is desirable but we can’t figure out a way to make it work within the constraints” to “X was never desirable anyway, you’re better off without it, we intentionally don’t allow X”. There’s nothing wrong with butting up against constraints and I hate that the demands of marketing end up polluting our language design knowledge base.
Thanks for the extra context! Mea culpa, my phrasing was ambiguous and doesn’t communicate exactly what I intended. I didn’t intend to make a comment on the history of the thing, but rather explain the perspective that makes the current state self-consistent.
Basically, if you think about Zig errors as weird sum-types, you’ll get a wrong mental and will try to make them do things that they can’t do. Instead, the much more useful mental model is “better typed integer error codes”.
I do claim that Zig errors are not a half-way towards polymorphic variants, but rather fully complete statically types integers.
I am also definitely not claiming that “fancy codes” are adequate for error-management related tasks. I actually don’t know how to manage errors in general. Like, Rust has precisely the same problem where, by default, you get AccessDenied without any indication of what path was being accessed. My personal strategy here is:
Result<E, T>sense. A compilation error is the same kind of outcome as normal compilation. The value of compilation task is(Vec<Diagnostics>, T).Ah, I see where you’re coming from now. That makes more sense than what I initially interpreted.
It’s interesting that if you follow those strategies then it kinda doesn’t make sense to have errorsets at all? If all the relevant information is outside the error code then why have an error code? Maybe I should just use
return error.erroreverywhere.This is the heart of my frustration with zig error handling. It’s almost a substantial improvement on everything else I’ve used, but then there is one missing piece and all the advice for dealing with that kinda sounds like “build your own error handling system and barely use zig’s error codes”.
I do use both of your strategies, but in both cases I find they really want to be anti-modular. Eg I end up with one big
Diagnosticunion and then if you only want to run the parser and not the whole compiler, the type signature still tells you that it might reportDiagnostic.linker_symbol_not_foundso you lose the value of the compiler checking that your error handling is exhaustive.And when I look at other peoples zig code almost everything only uses zigs error codes and doesn’t bother with the rest of the information. Because error codes are very convenient and blessed and as soon as you try to do more than just an error code there is a big ramp in painfulness.
I really do feel like what I want is zigs error system but with polymorphic variants. I’ll just be good and not return data that needs to be freed, I promise :)
Can you do it yourself? I am imagining API like this:
Basically:
@export,test, and@compileLogare side-effecting, so presumably you can cook something horrible with them? Can we just port https://github.com/dtolnay/linkme to zig?Hmm, I feel the nerdsnipe, but that does seem like the rest of the owl :D
Unrelated, I just wrote some code like:
So the contract is that if it returns error.Foo then foo_error is defined and vice versa. It’s a sum type! Just one that the compiler won’t notice when I mess it up.
(I mean, obviously the pragmatic solution is to make my own language :D)
This post makes the same mistake as part 1: it takes “nice to have” optimizations as a given, as something that has priority over everything else.
Nowhere does it discuss the obvious alternative: those “seemingly” correct optimizations are not, in fact, correct. And thus need to be abandoned.
As Ralf responded to that post, it takes those optimizations as a given because that is how existing compilers work, and the “obvious” alternative is far away from the one we live in today.
Discussions of pointer provenance always seem to have someone show up to make this argument that the whole system is just wrong and should be remade from scratch. But the people who make this argument never seem to have anything to say about the reasons compilers and language specs for C-like languages are this way, or how their preferred approach might tackle those problems.
It would be much more interesting to discuss what this alternative approach would actually look like. Presumably very few people would be satisfied with merely falling back to LLVM or GCC’s -O0 output- so which optimizations should compilers perform, and how can they be justified?
The entire post is a discussion of which of the three optimizations should be deemed incorrect.
Do you feel the same way about the strict aliasing rules? My understanding is that those were added back when C was competing for performance with Fortran. The performance penalty for “just doing what the programmer wrote” has only gotten larger since then.
I think the case for this with regards strict aliasing is actually stronger. That is, we know which optimizations need to assume strict aliasing and how much performance they win you - not a terrible amount, it turns out. That’s why GCC and Clang have -fno-strict-aliasing and why Linux uses it but is still plenty fast.
Provenance is much more central to how modern compilers work, its assumptions are so baked in that you can’t even tell these compilers “-fno-provenance” and see how much speed you lose.
That is, we have usable production compilers that don’t need strict aliasing, we don’t have any that don’t need provenance. So saying “strict aliasing is wrong” is a tenable position, and indeed Rust doesn’t have strict aliasing.
I spoke to Apple folks when their compiler team switched the default to strict aliasing. They reported that it made key workloads 5-10% faster and the fixes were much easier to do and upstream than I would have expected. My view of -fstrict-aliasing at the time was that it was a flag that let you generate incorrect code that ran slightly faster. They had actual data that convinced me otherwise.
In support of your argument, we do have wasm which has no UB. In some half-assed experiments I’ve found that no amount of optimizing wasm backend will make up for generating wasm with llvm O0, which seems like weak evidence that something in the c language semantics was necessary for better-than-O0 performance.
Poking through the generated code a lot of it is pointless stack loads/stores, which are impossible to eliminate unless you are allowed to make some assumptions about which pointers might alias local variables.
As a curmudgeonly old C guy, I hear you, but on the other hand, if I were to try to write that “everything else” down as a precise specification, I bet I couldn’t do it.
It feels like C is trying to be too many things at once. Maybe there should be a
#pragma do_what_I_saidfor code that needs to relate to the actual hardware. I mean, when I was doing low-level C seriously, even that optimization of hoistingi+jout of the loop would have been annoying. If I wanted it hoisted, I woud have hoisted it myself!Volatile is pretty close to that, no?
For the specific case of dereferencing individual pointer variables, yes. Though it’s still just an indirect hint and pretty easy to miss.
Assembly, inline or otherwise, is that pragma.
Which is tedious and annoying, which is why C was invented in the first place!
Last time I thought about this, I concluded that without provenance, a huge proportion of optimizations would be invalid, so to me it seems like much more than just a nice to have.
I personally find attempts to create new SSA-based backends to be a little misguided, especially if you’re not approaching it from a theoretically strong and novel research based approach. LLVM is slow and complicated almost as a necessity, and if you want a faster backend you have to cull a large amount of optimizations. You might as well use something like Cranelift at that point. Of course, these things are not binary and a I wish the authors the best of luck in creating something cool and useful.
A small nit-pick from the blog:
I think you’ll find this is the distinction between a compiler frontend and backend. The frontend produces the IR, which is optimized and turned into machine code (or assembly, object files, whatever) by the backend
Assuming it’s the same Yasser I know who’s writing an LLVM replacement, he doesn’t just have a theoretically strong approach, he has lots of input from “the guy” himself (Cliff Click, author of the Sea of Nodes paper in the 1990s, and the author of the SSA-based Hotspot Java Virtual Machine that billions of people rely on every day).
Yasser is young, and knows no limitations to his own abilities. In other words, he’s like most of us are – or were! – and while his success is not guaranteed, it’s people like him who know no limits that eventually will change the world. Hopefully for the better.
No, I don’t believe this to be true. LLVM is slow and complicated mainly because it was a early learning experience for its young author. And instead of incorporating the things that were already known, and the things learned in the writing of it, it simply accepted those inefficiencies as “unavoidable” and didn’t bother to correct any of them. Inertia is a great ally, and also a horrible foe.
That’s not really what I mean, I’m sure that Yasser is very talented. My contention is that creating a faster LLVM with comparable output is a research problem, and research problems should be approached scientifically. What is the hypothesis of this project? “A SSA backend based on sea of nodes can produce comparable output and significantly faster speeds”? This would be a significant advancement to Combining Analysis, Combining Optimizations, but as of right now there’s very little evidence to point to this being the case. I prefer smaller results that show promising signs, that can be used to build more significant results. Contrast this with the development of Copy and Patch by Haoran Xu, which is being developed into a remake of lua jit with significantly interesting results. I find his approach much more rigorous and also much more narrowly scoped and therefore much more interesting to follow.
As an aside, I think that some “pop” programmers have popularized the idea that all code is bloated and slow and needs to be completely re-written, and a lot of people have taken it on themselves to re-write huge portions of the stack themselves. And I think that’s great! As a fun personal project, but not the basis of a big serious project that I need to consider using. At least, I remain pretty skeptical of them early on.
I find this to be a miss-characterization of the development of LLVM, Chris Lattner was 25 but he was also a PhD student, and he wasn’t the only author - Vikram Adve, his doctoral advisor was 37. Yes, it was a learning experience for Chris, because it was his PhD thesis, so it was still quite a significant advancement over the state of the art.
Also, LLVM developed inertia because it had a license that made it appealing for Apple to fund the development of. I think it will be quite hard for any new project to achieve half of the features of LLVM, and you might as well therefore attempt to modify LLVM if you have ideas on how to improve it.
It’s not a research problem, it’s an engineering problem. He’s building something practical and useful to many, not exploring a novel concept and explaining it to a niche audience.
PhD students are notoriously terrible programmers, advisors even more so. They’re known for writing sloppy, hacky code that barely functions enough to get their thesis done. They embrace paradigms like Object-Oriented Programming which are popular among academics living in a bubble, but have been long outdated in the software engineering industry.
Writing an SSA backend that is faster and produces comparable output to LLVM is absolutely a research problem, and research does not have to be “exploring a novel concept and explaining it to a niche audience”.
Additionally, I will argue that all sufficiently sized engineering problems are research problems. You can’t just engineer your way through sufficiently large problems without research.
Again, I hope it does become practical and useful to many. But I think the foundations of the project make it unlikely to succeed in this front. I hope I’m wrong.
To me this feels just like something people say with no real evidence to back it up or disprove it. At best the evidence is anecdotal. I’ve met plenty of Drs of CS who write pretty good code, so I’m not sure why being a PhD would make you a poor coder besides other external factors like the stress of meeting the deadline for your thesis.
I’m not really trying to say anything about Chris Lattner, I’m just responding to your implied premise that being a PhD student means being a better programmer:
This is the initial claim being made and therefore the one that has the burden of proof. My comment only serves to point out that this claim is not a valid premise because it is not widely accepted as fact.
I wasn’t trying to imply that being a PhD student makes you a better programmer. I was just pushing back on the inexperience thing I guess.
This is very ungracious
It feels like saying “well Linus was 22 when he created Linux, and 22 year olds are notoriously terrible programmers”. Yes, there’s plenty to criticize from today’s POV, but that also just means the field has progressed, which is good.
Sure, the average Ph.D. student or their advisor is not a good programmer, but we’re talking about the people who created LLVM
Which Julia, Rust, Zig, Swift, and Odin all built on … It enabled a lot of open source to exist, which wouldn’t otherwise exist
If they were such horrible programmers, then surely it wouldn’t be a big deal to just ignore them and build something better. Why even build on top of something from these programmers?
Yet it will probably take a decade to build a good alternative (considering the targets, code gen speed, and runtime speed). And that will no doubt be done with lots of knowledge created by the LLVM project in the first place
LLVM also encouraged GCC to be more modular. AFAIK there were virtually no projects built on top of GCC – everything significant had to be maintained upstream in GCC because there were no public APIs
https://lobste.rs/s/jvruyj/tilde_my_llvm_alternative#c_8ardvl
We probably agree on a lot more of this than we disagree on. Any compiler back-end is a significant project of multiple man-years to do a half decent basic job of, let alone something that could replace LLVM. LLVM may not be optimal, but it has one significant advantage over most projects: It is available for use. That said, LLVM does have design flaws that have been pointed out over the years by people who have successfully built back-ends (i.e. not me – I’ve only done compiler front ends), and I have to assume that at least some portion of those criticisms are valid. Even Chris has admitted to various things that he wished he had done differently, so this isn’t some sort of blame game. I’m sure when Chris was building LLVM, a number of gatekeeping people were rolling their eyes at the audacity, but he pulled it off – criticisms notwithstanding. And I’m just trying avoid becoming one of those gatekeepers.
I agree with you on all the things stated here. I don’t intend to be unduly harsh or dismissive, and I do by no means wish to gatekeep the effort of anyone to do cool stuff.
https://home.cit.tum.de/~engelke/pubs/2403-cgo.pdf notes a lot of bottlenecks that are architectural rather than algorithmic. The ir encoding is memory-heavy, is expensive to build, and requires a lot of pointer-chasing. Extensibility at every stage is implemented by virtual function calls per ir node, which are expensive even if you don’t override them. The linker first builds a complete elf object and then immediately parses it again. Etc.
A new codebase that does the exact same optimizations but is architected for current hardware could potentially be a lot faster. Memory density and access patterns matter a lot more now than they did 20 years ago. Same for multi-threading.
Undoubtedly, I think this is an extremely good stand point for which to build a replacement (or large refactor) of LLVM. Thanks so much for the link!
Is there something particular about SSA, that you find misguided, or do you mean that not being novel would have a hard time competing with an already developed, industry-standard implementation?
What other alternatives are there? I know that GraalVM uses a graph-based IR, for example, which I found very interesting. Do you (or someone else) have perhaps some option on how these compare?
Nothing about SSA is misguided. You hit it on the head - without being novel or strongly researched based into LLVMs deficiencies (a good example of what this would look like is pointed to in another comment), you’re unlikely to compete with LLVM
Isn’t this just some sort of re-imagining of those interactive smalltalk VM+IDEs?
Everything is just some sort of re-imagining of something else. The devil is in the details though.
Smalltalk mixed persistent data and code together in a way that ended up being hard to maintain over time (eg schema migrations). Eve made all the data into a relational database, and those we know how to maintain.
I think airtable.com are the ones who actually nailed this though. Having an actual use case definitely helps design stay pragmatic.
This strikes me as a bold claim :)
(Anecdotally, the migrations I’ve done in Smalltalk have been about the same level of complexity as the migrations in DBMSs I’ve done. One important difference is lack of transactionality in (most) Smalltalks, meaning you generally perform migrations in a “stop the world” style as e.g. code is upgraded, or the image is (re)started, as required.)
It’s a shame GemStone never got an open-source clone (or, if it did, I never saw it). Seaside on Gemstone was amazing for web development. Every request ran in a separate transactional snapshot of the Smalltalk VM. This eventually hit scalability issues but they came much later than most other approaches.
I know there’s a smiley there, but this is an incredibly mild claim.
A sizeable majority of the entire worlds digital infrastructure runs on relational databases, at a level of scale and reliability that surpasses anything we’ve ever done with a smalltalk vm and that would blow the minds of a time-traveling programmer from the 80s.
I’m not saying there are no improvements to be made - I would be out of the job if databases were already perfect. But to the extent that we know how to maintain anything at all, our accumulated experience and expertise is overwhelmingly with relational databases. And we have by comparison barely any institutional knowledge of or experience with maintaining image-based systems.
So, I don’t know if this is really relevant to the overall discussion, but, I’ve learned as part of “received wisdom” that code in databases is different from data in databases, and is problematic - especially things like stored procedures, IIRC.
Why, specifically? I don’t know that. I haven’t used them. I can guess that versioning could be an issue? but then why isn’t this handled by migrations? Or is it that code stored in a DB requires more (or just different?) forethought than devs are used to thinking about with migrations? Maybe migration frameworks generally make assumptions that just don’t work well with stored procedures? I just don’t know. I wish I did, because I’ve often wondered about it. Input welcome from anyone who might see this reply.
random potential issues with storing behavior in DB I can imagine:
I have not written any production grade smalltalk so I am really not very familiar with this. In what way did smalltalk make maintaining schema migrations difficult over time?
Also, I skimmed through airtable.com and I have no idea what they are about :( so can you please tell me?
Airtable lets non-experts define databases and many views on their data with a spreadsheet-like interface.
There are lots of integrations for going further and making websites and apps using airtable as the database.
Mary Rose Cook gave an 8 minute talk at HYTRADBOI 2022 that shows off a lot of the features, and how it scales smoothly from spreadsheet-like to writing javascript.
https://www.hytradboi.com/2022/why-airtable-is-easy-to-learn-and-hard-to-outgrow/
DBMS migrations, Smalltalk migrations, and, say, Debian package upgrades are all roughly the same kind of thing. Similar considerations apply to each and each is about as tricky to get right as the others, in my experience.
In the sense that you no longer have code in files, but code in a richer structure (a database of sorts): yes. Smalltalk IDEs let you slice-and-dice the codebase to present it any way you want; they offer multiple simultaneous views into the code; they are live.
In almost every other sense: no. Smalltalk is not reactive; Smalltalk is not a logic or a relational language; Smalltalk has no events (though most system libraries have the observer pattern somewhat well-integrated).
There was some work from the VPRI STEPS project to make a reactive Smalltalk dialect 15 or so years ago. Eve looks a lot like the prototypes I saw of that. It’s a shame it doesn’t seem to have been updated since 2017, I’d like to see a real system that enables end-user programming like this.
Ahh so many good ideas in this lang. such a shame.
I think one of the big issues I see with things like this is just how different they are from everything else, and so you’re having to make things work on all fronts. Just building a properly reactive system is an endeavor, then add in what appears to be logic lang semantics, a whole editor, a ui library (that history widget), a debugger…
Yeah. It didn’t help that we didn’t know how to do anything of those things in the first place. It was a learning experience, for sure.
Hey that’s ok! Its great that you went out and tried something. I’m kinda in the opposite - I’ve had ideas like this, but end up learning what I can before actually going into the task, and its a lot to learn.
Tho I’m coming around to a fixed point to where what I want to make is fairly consistent and I’m about at the skill level I think I need. It takes a long time with self study.
A basic copy-and-patch approach seems like it would fail at implementing the query example from earlier in the article:
It seems like you might need to compile a template assuming multiple types of operations (table scan vs index lookup) and then pick the right template at runtime. That would get more complex when you start including joins (hash vs nested vs merge join), of which you could have an arbitrary number of combinations. Maybe there is a tractable set of basic queries where you could pre-compile and default to a slower interpreter for more complex queries. However, that might lead to some performance cliffs.
There is a database/web platform called Convex that is using v8 as it’s database engine. It’s explicitly not a SQL based engine and it’s using the Javascript parts of v8, not the WASM parts. I guess the developers have an ops background and really distrust databases/query optimizers so they decided to take a no SQL approach. One issue that came up was that they had to make promise execution more deterministic, which involved digging into the guts of V8. (I believe that they talked about it on this episode of software unscripted.) I imagine that there could be other issues like this if you decided to pull an engine out of the browser.
The copy-and-patch paper was reporting compilation times for tcp-h in the 100s of microseconds, so you would be able to just compile both query plans on the fly. You might not even bother having an interpreter at that point.
Hey, thanks for taking the time to reply! I enjoyed the post.
For some context, I’m a data engineer so I might be thinking of more complicated queries on average. Take the following query:
If we are talking about:
That might produce 4 tables * 2 lookup modes * 3 joins * 3 join modes = 96 different compilations, right? If we assume 100-500 microseconds per compilation target or 9.6-48 ms for the whole set of combinations. Or do you think that there would be enough information ahead of time to pare down the possibilities?
No, I mean when the query arrives you plan it and compile that plan. If copy-and-patch is as fast as claimed, it would only be a fraction of the planning time. No need to compile things in advance.
Okay, so there would be some parsing, query plan generation, and then a single copy-and-patch instance would be compiled. That makes sense. Thanks for clarifying!
I believe the author posted an answer below, but just to give you more details using your query as an example.
The query engine front-end, will start by parsing and analyzing the above query and emitting an AST.
The next step in the front-end is to transform the above AST to a logical query plan, this transformation will often output an AST but using different nomenclature,
SELECTbecomesProject,FrombecomesDatasourceorSubqueryin the case of nested queries for example.The above logical plan, will refer to columns by their indices; you can think of the execution step as consuming arrays that represent rows (classic OLTP) or batches of rows or columns (vectorized OLAP).
The logical plan is then taken in by the query planner, the query planner is the unit that will be responsible for using
TableScanorIndexScandepending on table statistics (like in Postgres) or just heuristics (OLAP systems don’t usually have indexes).The query planner will emit a physical plan, where the
Scanoperator is transformed to eitherTableScanorIndexScanand do the same for other operators (Joinfor example would be transformed toHashJoinorLoopJoin… it depends on the query engine here).There are other optimization steps that occur on the physical plan, but I won’t mention them here; at this step the query is finally ready for compilation.
Query compilation, involves emitting code that will operate on an input format that is already known, for example if the query engine uses Arrow then we can imagine query compilation as emitting; I’ll use Java here.
The above code, in the case of Apache Spark or Calcite, will be emitted to a raw string and then we’d invoke an embedded compiler like Janino to compile it to bytecode, load it into the JVM and run it against the inputs.
In the case of copy-and-patch, then what would happen is you would have a stencil that matches a query plan like the above, then it becomes a question of pattern matching. The compiler would then match the query plan against its own cache of plans, then load the bytecode for the stencil that describes the above query and patch the strings in the bytecode replace indices in the bytecode constant pool with
englishandusa.In the case of LLVM your IR would simply reference to raw literals that get patched with the above; for example Amazon Redshift does something similar where there is a cache of compiled query plans that get patched and run at runtime.
I find it interesting that the JVM based ecosystem is not referenced in the article because it has, in my opinion, more mature query compilers. To my knowledge all query compilers that are based on using the LLVM stack are closed source (SingleStore, HyPer, Umbra) and not that popular unlike Spark or Calcite.
The core idea of building such query compilers boils down to thinking in Futamura projects which can be described as building specialized interpreters using some knowledge about input data, this paper [1] delves into the subject and gives a very clear exposition. In [2] there is a very nice and high level exposition of how query engines work with some effort you can add query compilation as a last step for fun :).
[1] https://www.vldb.org/pvldb/vol7/p853-klonatos.pdf [2] https://howqueryengineswork.com/
I’m so glad to see this, I’ve been throwing around the idea for an “LLVM for relational databases” for a while now, partly due to the seemingly backwards workflow of pasting source code from one language into string-literals of another language and losing all semblance of type-safety/IDE integration without editor-specific plugins etc.
I love sqlc but it still has drawbacks, and my mental model for why we keep running into this discussion of “raw sql vs orms” is there’s an abstraction layer bump/mismatch. SQL is a high level language, very high level, application/business logic code is “lower” level and a lot of these tools that work with SQL (sqlc, your average orm, prisma, etc) are essentially going up the abstraction ladder then back down again. As in, you write some ORM code, it gets “compiled upwards” into SQL which is then “compiled downwards” into a query plan. And of course you usually want to control your SQL at some point to fine-tune/control/optimise it (because “declarative” is a lie) so you bin the ORM and now spend time wiring up the outputs from flat tables (even if you want 2-3 layers of data thanks to a handy join) and then spend more time unwrapping that flat list back into its structured form.
And that being said, I really like the end-user experience (ignoring optimisation) of tools like Prisma, Ent, etc. they let me build very fast and the mental model of my data fits perfectly first time. But as soon as I need to break out of that cosy bubble, time has to be spent doing a lot of manual work.
And on top of that my general thesis is that SQL is stuck in 1973 and there’s almost zero “competition” or flourishing ecosystem of new language design simply due to the highly coupled nature of “I need to speak to Postgres because that’s our database and will forever be our database” so you’re locked in to the weird non-spec-compliant Postgres flavour of SQL forever.
This didn’t happen with application languages partly because of things like LLVM, JVM, etc. Someone, a fresh, young, innovative and smart person can come along and write a brand new language with interesting and challenging ideas to be critiqued, experimented with and maybe eventually deployed without needing to worry too much about all the CPU architectures and system details. They can just target LLVM and bloom a new idea into the world to be shared and enjoyed by others.
I suppose what I want is a baseline platform, like LLVM, on which smart people can try new things that actually have a chance of being used on real projects. There are plenty of toy databases out there with new ideas, that paper from Google with pipe-like syntax for SQL was awesome! But I’ll probably never get to use that because basically every real-world product is going to be using an incumbent database and be locked in to their flavour of SQL. Yes, you can “transpile” some interesting new language to SQL but then you run in to the original problem of “I need control over the actual underlying SQL query” and it all breaks down because you’re going up the abstraction ladder rather than down.
An ill-formed thought I suppose but hopefully you catch my drift!
What are the most common extensions that you need Postgres for? i.e. the things that won’t run on MySQL. Is it things like
Is it totally foolhardy to try to be portable across Postgres and MySQL? I mean I guess that is what an ORM gives you, but ORMs are leaky abstractions, and you will often need to drop down.
I know that Python’s sqlite binding is badly hobbled by this ill-conceived idea to be portable across databases. In particular I remember dealing with silly hacks where it would textually paste SQL code around your code for transactions, so that sqlite would match the semantics of other databases. But this is brittle and can introduce performance problems
Going back to the original comment, I think the LLVM analogy only goes so far.
Because LLVM is not a stable API – it’s NOT a VM (which it says on the home page!) It’s a C++ library which can change at any release.
I think the problem is that MySQL has its own VM, and sqlite has its own VM, etc. And then the SQL layer is the common thing on top of that.
So it’s hard to introduce another VM layer – internally it would make sqlite a lot more complex, with no real benefit to sqlite itself
It’s similar to the problem of “why don’t we export a stable AST for every programming language?” The text representation is actually the stable thing; the internals can vary wildly between releases
I do agree that LLVM had a transformative effect on the language ecosystem though, with C++ Julia Rust Swift all using it.
The postgres thing was just an example, but I’ve consistently run into this over my career, and it’s not really about portability of software (I’ve never once worked in a team that actually switched databases - mostly because unless you have really specific needs, most tech teams will just pick their favourite, which in my experience is often Postgres. I think the “ability to switch database” isn’t a great argument tbh because it’s akin to rewriting your entire product in a different language, a hard sell for any business) it’s more the transferability of knowledge. I consistently forget `, “ and ’ are semantically different, I forget that some DBs want table.column or namespace.table.column format, I forget that Postgres does conflict clauses differently to SQLite and MySQL (from what I remember, postgres requires you to explicitly specify each conflict column, SQLite lets you just say “on any conflict, do this”)
One more (arguably, bad) example is migrations: I run an open source project that users deploy, it supports the 3 main DBs of the world, purely because it uses Ent which (the way I have it set up) dynamically nudges the schema to where it needs to be when the app boots up. I’d love to move to a more resilient migration strategy where I store a historical log of files with the “alter table” stuff in it, but for that I’d need to maintain 3 separate folders of slightly different generated migrations for each DB due to all the minor syntax differences. A bit of a pain tbh, and something that keeps me on the dynamic approach (which works well 99% of the time and runs into impossible changes 1% of the time, which I must manually fix and let users know how to fix for new releases, NOT ideal!)
But tbh, devil’s advocate: you could all argue those are just a skill issue on my part and I should learn my tools better!
But yes, you’re right, the LLVM comparison is half the way there. All these databases implement their own virtual layers (CRDB does a key-value store with an engine that translates SQL into KV scans, SQLite has some bytecode language, not sure what postgres does)
I don’t think this utopian idea of mine would ever work with incumbent products (if I ever get around to building it) but I do think there’s a gap in the market for a “not the fastest cutting edge DB in the world, but one with a really really nice developer experience that’s perfect for simple CRUD apps”
Though to clarify, my idea is not to literally use LLVM itself, but to construct a sort of new paradigm where there’s an LLVM-for-relational-queries that sits in the middle of some interface (be it SQL or a type-safe codegen solution for your app code) and the actual data store. But then what you have is a relatively neat relational data store and a (ideally) stable IR layer upon which anyone can build a cool new language without needing to understand/optimise the horrors of a relational algebra implementation!
A pipe dream. But perhaps something I’ll get to in my free time one day!
This pretty much exists now for OLAP databases between parquet, arrow, datafusion, and substrait. Check out this playlist - https://www.youtube.com/watch?v=iJhRbDFJjbg&list=PLSE8ODhjZXjZc2AdXq_Lc1JS62R48UX2L.
OLTP databases aren’t there yet. Probably because the storage isn’t pure data but mixes in things like mvcc and indexes, so it’s harder to find common ground between databases.
Oh neat, I’ve not looked into datafusion this looks in the same ballpark for sure! I did try to get my head around substrait but just couldn’t figure it out. Thanks for sharing!
I do feel like this may be one of those situations where my best way to learn is to just build something myself and run into all these complexities on my own then refer back to the prior art!
I’ve been saying for a while that I would love to see a Warren Abstract Machine for SQL. Maybe it wouldn’t be the most powerful or performant SQL database, but having a platform for language development seems like it would be very helpful.
This is one of the unique powers of
declarativenominal type systems, to create a zero-runtime-cost wrapping type. I miss this ability in TypeScript where two aliases to one type, or two interfaces with equivalent requirements, are automatically interchangeable.It’s not really about type systems so much as control over layout. Eg julia is dynamically typed but
struct Foo; i::Int64; endis zero-runtime-cost.The problem with javascript is universal layout. You can make
{i: 42}almost zero-overhead with object shapes, but there’s no way to store it inline in an array or another object, so you’ll always incur the extra heap allocation.Whereas in Julia you can make an
Arrray{Foo}and it will have exactly the same layout asArray{Int64}.In the course of writing this I noticed I should have said “nominal type system” instead of declarative:
I want to make sure we’re talking about the same thing… I think I take your point that the type system is not what provides the zero runtime cost aspect, but I think nominal typing is a factor in providing the safety benefit in these types. Here is what I mean:
In TypeScript’s structural typing, I can write:
There’s no error on the last line, because a ProductID provides guarantees that satisfy all of PersonID’s requirements. But I want an error, because I don’t intend them to be substitutable. Structurally identical objects are evaluated the same by all types. To get this safety I would need to make unique structure somehow.
Whereas in Swift’s nominal typing, the final line will fail because a PersonID is not a ProductID. Same layout does not mean type checks treat them the same, and that is useful.
Like this?
You might still worry about collisions between keys in different libraries. Clojure had a neat solution to this - the symbol
::fooexpands to:current-namespace/foowhich is not equal to:some-other-namespace/foo. So I might end up writing{::product-id "blah"}which expands to{:net.scattered-thoughts.cool-app/product-id "blah"}.In the limit, there isn’t much difference between nominal and structural typing, except that nominal types hide their unique value and structural types have to make it explicit. But making the unique value explicit can make some things simpler, like clojures automatic serde (https://www.scattered-thoughts.net/writing/the-shape-of-data).
This comment is verging on a long “well actually”, but I do think that structural types are underrated because of this specific confusion.
I’ve been working on a gradually structurally typed language where this can all be zero runtime cost too, like julia. You’d write:
And a value of type
product-id-typeis exactly the same size as a string. The key is stored in the type itself.Yeah, Clojure’s namespacing of map keys instead of whole product types is a great idea. Thanks for all the detail! I don’t have much to add but I appreciate it. Also count me a fan of any language that leans on gradual typing and kebab case.
I think this is reflective of a real culture difference. For someone who is used to systems languages, a dense bit vector is a standard tool, as unremarkable as using a hashmap. One of the reasons that optimization is so frustrating in javascript is that many other standard tools, like an array of structs, are just not possible to represent. It feels like working with one hand tied behind your back. When you’re used to being able to write simple code that produces efficient representations, it’s hard to explain to someone who isn’t used to that just how much harder their life is.
Another reason is that the jit is very hard to reason about in a composable way. Changes in one part of your code can bubble up through heuristics to cause another, seemingly unrelated, part of your code to become slower. Some abstraction method might be cheap enough to use, except on Tuesdays when it blows your budget. Or your optimization improves performance in v8 today but tomorrow the heuristics change. These kinds of problems definitely happen in c and rust too (eg), but much less often.
Even very simply things can be hairy. Eg I often want to use a pair of integers as the key to a map. In javascript I have to convert them to a string first, which means that every insert causes a separate heap allocation and every lookup has to do unpredictable loads against several strings. Something that I would expect to be fast becomes slow for annoying reasons that are nearly impossible to work around.
It’s definitely possible to write fast code in javascript by just turing-tarpitting everything into primitive types, but the bigger the project it gets the larger the programmer overhead becomes.
I definitely recognize the value of a familiar language though, and if you can get within a few X of the performance without too much tarpit then maybe that’s worth it. But I wouldn’t volunteer myself for it. Too frustrating.
That line also struck me in particular. Using a Uint8Array as a bitvector is definitely something I’ve thought of, if not even implemented, a few times in JS. As you said, JS seems to actively work against you when you’re trying to optimize your code in ways that would work in other languages, so it disincentivizes you from trying to reason about performance.
There’s a lot to be said about consistency in object representation and opimizations in “dynamic” languages. OCaml is renowned to be a fast language not because it does crazy whole-program optimizations (the compiler is very fast precisely because it does no such thing) or because its memory representation of objects is particularly memory efficient (it actually allocates a lot; even
int64s are heap allocated!), but because it is straightforward and consistent, which allows you to reason about performance very easily when you need it.The thing about this is that it doesn’t even feel like it’s “optimization” – as much as non-pessimization of how one would store such data. A single block of memory seems straightforward, with the more complicated way being to have a bunch of separate heap allocated objects that are individually managed. That’s what ends up feeling nice to me about the languages JS is being contrasted with here: I actually just write the basic implementation that is a word-for-word reflection of the design and get a baseline performance right off the bat that lets me go for a long time (usually indefinitely) before having to consider ‘optimizing’. The productivity boost offered alongside not giving that seems to me like it would have to be really really high to be worth the tradeoff, and personally I haven’t felt like JS delivers on that.
That’s true, but there’s a big tradeoff in flexibility there. If you want to store these objects in continuous memory now your language needs to know how big these objects are and may be in the future, which effectively means you need a type system of some sort. If you want to move parts of this objects out of it then you need to explicitly copy them and free the original object, so either you or your language needs to know where these objects are and who they belong to. If you want to stack-allocate them now your functions also need to know how big these objects are, and if you want the functions to be generic you’ll have to go through a monomorphization step, which means your compiler will be slower.
I’m not saying these properties are not valuable, but the complexity needs to lie somewhere. JS trades the ease of not having to keep track of all of this with a more complicated and “pessimized” memory model, while languages that let you have this level of control are superficially much harder to use.
Makes sense, but I’ve just found that what I pay in each of those scenarios comes up when each of those concretely happens and is often / usually a payment that seems consistent with the requirement and can be done in a way that’s pertinent to the context (I tend to not try to address all of those upfront before they actually happen, at this point). However, the performance requirement tends to either be always present or even if not present just be a mark of quality or discovery regardless (eg. if I can just have more entities in my game in my initial prototype I may discover kinds of gameplay I wouldn’t have otherwise).
I think I don’t consider those tradeoffs you’re listing very negatively because the type system usually comes with the language I use and I consider this to be a baseline for various sorts of productivity and implementation benefits at this point (which seems to also be demonstrated in the case of JS by how much TypeScript is a thing – it’s just that TypeScript makes you write types but doesn’t give you the performance and some other benefits that would’ve come with them elsewhere). Similar with stack allocation and value types.
I’m not seeing how the other side of that tradeoff is ‘flexibility’ specifically though, because my experience is that the type system lets me refactor to other approaches with help from the language’s tooling, and so far the discussion has been about how the JS approach is actually inflexible regarding that.
Re: ‘the complexity needs to lie somewhere’ – the point is that in the base case (eg. iterating an array of structs of numbers) there is actually less essential complexity and so it may not have to lie anywhere. It’s just that JS inserts the complexity even when it wasn’t necessary. It seems more prudent to only introduce the complexity when the circumstances that require it do arise, and to orient the design to those requirements contextually.
All that said, I’m mostly just sharing my current personal leaning on things that leads to my choice of tools. I just feel like I wasn’t actually that much more productive in JS-like environments, maybe even less so. On the whole I’m a supporter of trying various languages and tools, learning from them and using whatever one wants to.
I agree with everything you said. I’m just not sure that we should be defaulting to writing everything in Rust (as much as I like it) or some other similar language. I think I wish for some sort of middle ground.
Agreed on that. I definitely don’t think that the current languages – on any of the points in this spectrum – are clear winners yet, or that it’s a solved space. I avoided mentioning any particular languages on the ‘more control’ side for that reason.
Aside: Lately I’ve been playing with C along with a custom code generator (which generates container types like
TArrayfor each T and also ‘trait’ functions likeTDeallocandTCloneetc. that recurse on fields) of all things. Far from something I’d recommend seriously for sure, just a fun experiment for side projects. It’s been a surprising combination of ergonomics and clarity. Turns out__attribute__((cleanup))in GCC and clang allow implementingdeferas well. Would be cool to get any basic attempt at flow-sensitive analysis (like borrow-checking) working in this if possible, but I’ve been carrying on with heavy use of lsan (runs every 5 seconds throughout app lifetime) and asan during development for now.This isn’t TigerBeetle, right? It’s just on their channel because… it was at a conference they hosted?
Yes, it’s from the Systems Distributed playlist.
Some later papers that seem interesting:
https://ir.cwi.nl/pub/28649/28649.pdf
https://core.ac.uk/download/pdf/301638222.pdf