It’s great to see Sidekiq still going strong after all these years! The iteration feature is really interesting, and I can think of many situations where it would have come in very handy but I had to reach for a more complicated solution. Don’t think I’ve seen that in similar libraries.
Every big Rails team I have worked on ends up building their own janky version of it (I’ve built a couple), very nice to have Sidekiq provide an implementation.
Every place I’ve ever worked I’ve always had a machine capable of running the test suite magnitudes faster than CI. I used to jokingly tell my colleague “What if I could just tell GH the tests pass” while waiting for them to finish
My workflow is typically: run all the tests locally (very fast), push up code then wait an 10 extra minutes just to confirm the same thing (that all the tests pass).
The main argument against is that even if you assume good intentions, it won’t be as close to production as an hosted CI (e.g. database version, OS type and version, etc).
Lots of developers develop on macOS and deploy on Linux, and there’s tons of subtle difference between the two systems, such as case sensitivity of the filesystem, as well as default ordering just to give an example.
To me the point of CI isn’t to ensure devs ran the test suite before merging. It’s to provide an environment that will catch as many things as possible that a local run wouldn’t be able to catch.
To me the point of CI isn’t to ensure devs ran the test suite before merging.
I’m basically repeating my other comment but I’m amped up about how much I dislike this idea, probably because it would tank my productivity, and this was too good as example to pass up: the point of CI isn’t (just) to ensure I ran the test suite before merging - although that’s part of it, because what if I forgot? The bigger point, though, is to run the test suite so that I don’t have to.
I have a very, very low threshold for what’s acceptably fast for a test suite. Probably 5-10 seconds or less. If it’s slower than that, I’m simply not going to run the entire thing locally, basically ever. I’m gonna run the tests I care about, and then I’m going to push my changes and let CI either trigger auto-merge, or tell me if there’s other tests I should have cared about (oops!). In the meantime, I’m fully context switched away not even thinking about that PR, because the work is being done for me.
You’re definitely correct here but I think there are plenty of applications where you can like… just trust the intersection between app and os/arch is gonna work.
But now that I think about it, this is such a GH-bound project and like… any such app small enough in scope or value for this to be worth using can just use the free Actions minutes. Doubt they’d go over.
any such app small enough in scope or value for this to be worth using can just use the free Actions minutes.
Yes, that’s the biggest thing that doesn’t make sense to me.
I get the argument that hosted runners are quite weak compared to many developer machines, but if your test suite is small enough to be ran on a single machine, it can probably run about as fast if you parallelize your CI just a tiny bit.
With a fully containerized dev environment yes, that pretty much abolish the divergence in software configuration.
But there are more concern than just that. Does your app relies on some caches? Dependencies?
Where they in a clean state?
I know it’s a bit of an extreme example, but I spend a lot of time using bundle open and editing my gems to debug stuff, it’s not rare I forget to gem pristine after an investigation.
This can lead me to have tests that pass on my machine, and will never work elsewhere. There are millions of scenarios like this one.
I was once rejected from a job (partly) because the Dockerfile I wrote for my code assignment didn’t build on the assessor’s Apple Silicon Mac. I had developed and tested on my x86-64 Linux device. Considering how much server software is built with the same pair of configurations just with the roles switched around, I’d say they aren’t diminished enough.
Was just about to point this out. I’ve seen a lot of bugs in aarch64 Linux software that don’t exist in x86-64 Linux software. You can run a container built for a non-native architecture through Docker’s compatibility layer, but it’s a pretty noticeable performance hit.
One of the things that I like having a CI is the fact that it forces you to declare your dev environment programmatically. It means that you avoid the famous “works in my machine” issue because if tests works in your machine but not in CI, something is missing.
There are of course ways to avoid this issue, maybe if they enforced that all dev tests also run in a controlled environment (either via Docker or maybe something like testcontainers), but it needs more discipline.
This is by far the biggest plus side to CI. Missing external dependencies have bitten me before, but without CI, they’d bite me during deploy, rather than as a failed CI run. I’ve also run into issues specifically with native dependencies on Node, where it’d fetch the correct native dependency on my local machine, but fail to fetch it on CI, which likely means it would’ve failed in prod.
This is something “local CI” can check for. I’ve wanted this, so I added it to my build server tool (that normally runs on a remote machine) called ding. I’ll run something like “ding build make build” where “ding build” is the ci command, and “make build” is what it runs. It clones the current git repo into a temporary directory, and runs the command “make build” in it, sandboxed with bubblewrap.
The point still stands that you can forget to run the local CI.
This was a very Ruby-centric take (not surprising, the author is on the Rails core team). It’s true that if you’re not a CDN then HTTP/2 probably has little impact on you as a developer…unless you use bi-directional streaming.
HTTP/2 adds bi-directional streaming, which moves HTTP from a serial request-response model to a concurrent request-response model. This unlocks way richer API design in browser-less environments like backend services. It’s probably most often utilized in gRPC services that use client and/or server streaming.
As a longtime-sometimes Rubyist, I get why there’s not a lot of love for bi-directional streaming. But it’s disappointing because it means that there are better languages & frameworks out there for certain types of backend APIs.
Yes, everything published on my blog should be read from a Ruby point of view and I believe the intro does a good enough job at explaining the scope of my opinion:
From time to time, either online or at conferences, I hear people complain about the lack of support for HTTP/2 in Ruby HTTP servers, generally Puma.
And yes, gRPC is probably the only reason you’d want full HTTP/2 support in a Ruby server, but the official grpc bindings for Ruby are so terrible that I wouldn’t recommend to try to do gRPC with Ruby to my worst enemy, and much simpler protocols like Twirp would give you 90% of the capabilities with much less headaches.
The wording of the blog post confused me, because in my mind “FFI” (Foreign Function Interface) usually means “whatever the language provide to call C code”, so in particular the comparison between “the function written as a C extension” and “the function written using the FFI” is confusing as they sound like they are talking about the exact same thing.
The author is talking specifically about a Ruby library called FFI, where people write Ruby code that describe C functions and then the library is able to call them. I would guess that it interprets foreign calls, in the sense that the FFI library (maybe?) inspects the Ruby-side data describing their interface on each call, to run appropriate datatype-conversion logic before and after the call. I suppose that JIT-ing is meant to remove this interpretation overhead – which is probably costly for very fast functions, but not that noticeable for longer-running C functions.
Details about this would have helped me follow the blog post, and probably other people unfamiliar with the specific Ruby library called FFI.
Replying to myself: I wonder why the author needs to generate assembly code for this. I would assume that it should be possible, on the first call to this function, to output C code (using the usual Ruby runtime libraries that people use for C extensions) to a file, call the C compiler to produce a dynamic library, dlopen the result, and then (on this first call and all future calls) just call the library code. This would probably get similar performance benefits and be much more portable.
I would guess because that would require shipping a C compiler? Ruby FFI/C extensions are compiled before runtime; the only thing you need to ship to prod is your code, Ruby, and the build artifacts.
Yeah, that’s why I wrote “predominantly”. Also, for such a localised JIT, a second port to aarch64 is not that hard. You just won’t have an eBPF port falling out of your compiler (this is absurd for effect, I know this isn’t a reasonable thing).
Note that the point here is not to JIT arbirary Ruby code, which is probably quite hard, but a “tiny JIT” to use compilation rather than interpretation for the FFI wrappers around external calls. (In fact it’s probably feasible, if a bit less convenient, to set things up to compile these wrappers ahead-of-time.)
the zero-byte can be omitted if the data is intended to be used only with languages which do not use null-terminated strings
Seems like a bad idea to me. It’s too easy to have code that works until it doesn’t because the provided data stopped adding the extra null byte. If it’s really valuable to save that one byte, then it probably should be a different string TYPE_.
Also this is simpler than messagepack, but not that much simpler. The TYPE_STRINGREF is a nice touch
though, particularly for the classic “list of objects” JSON structure.
It doesn’t seem complicated. Use SQLite for your single user desktop application, use PostgreSQL for a multi-user server, and don’t expect that the default configuration is well suited for your machine or your workload. That last is true for any data store (including filesystems, not just databases).
This was true, at least for ruby on rails about one year ago. But recently rails developers merged some PRs to leverage the concurrency features of the sqlite database. So, this may no longer be true.
I personally think the above advice is still sound. I merged these SQLite improvements to Rails, because why not, but I’m personally very skeptical and wary of the current SQLite hype (if not over-hype).
If you are dead certain you’ll never need to span your application on more than one server, SQLite may make sense, but even then it might not be the most efficient solution.
I see how not having to administrate a PostgeSQL or MySQL service is attractive, but you’ll still have to sort out backups and such. You can’t just copy the files you have to lock the DB while doing so etc.
Also even with WAL etc, SQLite performs very well with reads, but not so well if your application does frequent writes, and with Ruby’s GVL it can happen that you cause the write lock to be held for much longer than you’d want.
So there’s some use cases where SQLite makes perfect sense (single machine, read heavy apps), but to me it’s the exception not the rule. I’d definitely consider SQLite if I had a use case that call for it, but I’d never make it part of my default stack (as in the stack I use when requirements aren’t super clear yet).
This is a very interesting article. I was originally taken aback by the initial “Not IO bound” comment, but pointing out that our current understanding of IO is actually conflated with the OS scheduling of threads was very on point. I hadn’t considered that before. I think my reaction still stands though, but in a pedantic way. Looking at:
YJIT only speeds up Ruby by 2 or 3x
and
Like Discourse seeing a 15.8-19.6% speedup with JIT 3.2, Lobsters seeing a 26% speedup, Basecamp and Hey seeing a 26% speedup or Shopify’s Storefront Renderer app seeing a 17% speedup.
I still feel that if a component sees a 2-3x perf increase and that translates to (1.15x-1.27x) improvement that it’s a significant component (and well worth optimizing), but it isn’t the dominant/limiting factor.
Towards the end of the article Jean gets into some specific numbers regarding “truly IO bound” being 95% and “kinda” being 50%. I asked him on Mastodon about them. https://ruby.social/@byroot/113877928374636091. I guess in my head “more than 50%” would be what I would classify as “IO bound.” Though I’ve never put a number to it before.
Samuel shared this in Ruby core chat, and (spoiler) that’s actually one trick for debugging performance. They want to answer the question “Is this code worth optimizing” i.e. “if we made this code 2x faster…would anyone care.” Because if you make something 100x faster that only accounts for 1% of your total time, people aren’t really going to notice.
So they can’t arbitrarily make code faster, but they CAN make code arbitrarily slower. So the program simulates a speedup of one code section by making all the other code slower to report if it’s worth optimizing it or not. An all around interesting talk. It’s very aprochable as well
It would be interesting to have some kind of an IO backend where you could simulate a slowdown. I.e. perform the query on the database and time it, then sleep for some multiplier of that time before returning. It would (in theory) let you put a number to how much your app is affected by (database) IO. If you set a 2x multiplier and you see requests take 2x as long…then you’re approaching 100% IO bound.
The linked GVL timing gem is new and interesting. Overall, thanks for writing all this down. Great stuff.
I guess in my head “more than 50%” would be what I would classify as “IO bound.”
Perhaps I should have written about that in my conclusion, but ultimately IO-bound isn’t perfect to describe all I want to say.
In a way it’s a good term, because the implication of an IO-bound app, is that the only way to improve its performance is to somehow parallelize the IOs it does, or to speedup the underlying system it does IOs with.
With that strict definition, I think YJIT proved that it isn’t the case, given it proved it was able to substantially speedup applications.
A more relaxed way I tend to use the IO-bound definition, in the context of Ruby applications, is whether or not you can substantially increase your application throughput without degrading its latency by using a concurrent server (typically Puma, but also Falcon, etc).
That’s where the 50% mark is important. 50% IO implies 50% CPU, and one Ruby process can only accommodate “100%” CPU usage. And given threads won’t perfectly line up when they need the CPU, you need substantially more than 50% IO if you wish to process concurrent requests in threads without impacting latency because of GVL contention.
So beyond trying say whether apps are IO-bound or not, I mostly want to explain under which conditions it makes sense to use threads or fibers, and how many.
50% IO implies 50% CPU, and one Ruby process can only accommodate “100%” CPU usage. And given threads won’t perfectly line up when they need the CPU, you need substantially more than 50% IO if you wish to process concurrent requests in threads without impacting latency because of GVL contention.
Are you comparing multiple single threaded processes to a single multithreaded process here? Otherwise, I don’t understand your logic.
If a request takes 500msec of cpu time and 500msec of “genuine” io time, then 1 request per second is 100% utilization for a single threaded server, and queue lengths will grow arbitrarily. With two threads, the CPU is only at 50% utilization, and queue lengths should stay low. You’re correct that there will be some loss due to requests overlapping, and competing for CPU time, but it’ll be dominated by the much lower overall utilization.
In the above paragraph, genuine means “actually waiting on the network”, to exclude time spent on CPU handling the networking stack/deserializing data.
P.S. Not expressing an opinion on “IO bound” it’s not one of my favorite terms.
Are you comparing multiple single threaded processes to a single multithreaded process here?
Yes. Usually when you go with single threaded servers like Unicorn (or even just Puma configured with a single thread per process), you still account for some IO wait time by spawning a bit more processes than you have CPU core. Often it’s 1.3 or 1.5 times as much.
I don’t think there’s anything special about 50% CPU. The more CPU time the worse, but I don’t think anything changes significantly at that point, I think it’s going to be relatively linear relationship between 40% and 60%.
You’re going to experience some slowdown (relative to multiple single-threaded processes), as long as the arrival rate is high enough that there are ever overlapping requests to the multithreaded process. Even if CPU is only 1/4 of the time, your expectation is a 25% slowdown for two requests that arrive simultaneously.
I think, but am not sure, that the expected slowdown is “percent cpu time * expected queue length (for the cpu)”. If queue length is zero, then no slowdown.
You can achieve some of that with a combination of cgroups limiting the database performance and https://github.com/Shopify/toxiproxy for extra latency/slowdown on TCP connections.
If anyone’s curious, this is now live in prod. I’ll remove it roughly tomorrow morning (depends on my social calendar). I pasted a few lines from the log to the PR so it’s obvious there’s no personal info going to byroot.
I know I’ve said this before, but one of the reasons I find maintaining the Lobsters codebase rewarding is that it’s landed in a sweet spot where it handles real-world complexity and some scale, but is small, standard-style, and with few enough dependencies that devs can find their way around quickly. People have used it for corporate training on writing good tests, to learn or experiment with Rails, to improve Ruby itself, to design a database, to crib small examples from. We’ve benefited hugely from the generosity of the projects we’re built on and we get to share onwards in turn.
One thing that I found interesting is that many changes in each Ruby release would be considered a big no for any other language (for example the “Keyword splatting nil” change) since the possibility of breaking existing code is huge, but Ruby community seems to just embrace those changes.
I always think about the transition between Python 2 and 3 that the major change was adopting UTF-8 and everyone lost their minds thanks to the breakage, but Ruby did a similar migration in the version 2.0 and I don’t remember anyone complaining.
I am not sure if this is just because the community is smaller, if the developers of Ruby are just better in deprecating features, or something else. But I still find interesting.
Ruby’s version of the Python 2 to 3 experience (by my memory) came years earlier, going from 1.8 to 1.9. It certainly still wasn’t as big of an issue as Python’s long-lingering legacy version, but it was (again, my perception at the time) the Ruby version that had the most lag in adoption.
Yes, and it was very well managed. For example, some changes were deliberately chosen in a way that you had to take care, but you could relatively easy write Ruby 1.8/1.9 code that worked on both systems.
The other part is that Ruby 1.8 got a final release that implemented as much as the stdlib of 1.9 as possible. Other breaking things, like the default file encoding and so on where gradually introduced. A new Ruby version is always some work, but not too terrible. It was always very user centric.
It was still a chore, but the MRI team was pretty active at making it less of a chore and getting important community members on board to spread knowledge and calm the waves.
Honestly, I think Ruby is not getting enough cred for its change management. I wish Python had learned from it, the mess of 2 vs 3 could have been averted.
Interesting POV. As a long-time Rubyist, I’ve often felt that Ruby-core was too concerned with backwards compatibility. For instance, I would have preferred a more aggressive attempt to minimize the C extension API in order to make more performance improvements via JIT. I’m happy to see them move down the path of frozen strings by default.
One thing that I found interesting is that many changes in each Ruby release would be considered a big no for any other language (for example the “Keyword splatting nil” change) since the possibility of breaking existing code is huge, but Ruby community seems to just embrace those changes.
Like others already said, the Ruby core team stance is almost exactly the opposite: it is extremely concerned with backward compatibility and not breaking existing code (to the extent that during discussion of many changes, some of the core team members run grep through the codebase of all existing gems to confirm or refute an assumption of the required change scale).
As an example, the string literal freezing was discussed for many years, attempted before Ruby 3.0, was considered too big a change (despite the major version change); only pragma for opt-in was introduced, and now the deprecation is introduced in the assumption that the existence of pragma prepared most of the codebases for the future changes. This assumption was recently challenged, though, and the discussion is still ongoing.
Keyword splatting nil change might break only the code that relies on the impossibility of the nil splatting, which is quite a stretch (and the one that is considered acceptable in order to make any progress).
Keyword splatting nil change might break only the code that relies on the impossibility of the nil splatting, which is quite a stretch (and the one that is considered acceptable in order to make any progress).
This seems like really easy code to write and accidentally rely on.
def does_stuff(argument):
output = do_it(argument)
run_output(output) # now `output` might be `{}`
rescue StandardError => e
handle(e)
end
def do_it(arg)
splats(arg)
end
If nil was expected but was just rolled up into the general error handling, this code feels very easy to write.
Well… it is relatively easy to write, yes, but in practice, this exact approach (blanket error catching as a normal flow instead of checking the argument) is relatively rare—and would rather be a part of an “unhappy” path, i.e., “something is broken here anyway” :)
But I see the point from which this change might be considered too brazen. It had never come out during the discussion of the feature. (And it was done in the most localized way: instead of defining nil.to_hash—which might’ve been behaving unexpectedly in some other contexts—it is just a support for **nil on its own.)
I have to doubt that. It’s extremely common in Python, for example, to catch ‘Exception’ and I know myself when writing Ruby I’ve caught StandardError.
I don’t mean catching StandardError is rare, I mean the whole combination of circumstances that will lead to “nil was frequently splatted there and caught by rescue, and now it is not raising, and the resulting code is not producing an exception that would be caught by rescue anyway, but is broken in a different way”.
Like others already said, the Ruby core team stance is almost exactly the opposite: it is extremely concerned with backward compatibility and not breaking existing code (to the extent that during discussion of many changes, some of the core team members run grep through the codebase of all existing gems to confirm or refute an assumption of the required change scale).
But this doesn’t really matter, because there are always huge proprietary codebases that are affected for every change and you can’t run grep on them for obvious purposes. And those are the people that generally complain the most about those breaking changes.
Well, it matters in a way that the set of code from all existing gems covers a high doze of possible approaches and views on how Ruby code might be written. Though, of course, it doesn’t exclude some “fringe” approaches that never see the light outside the corporation dangeons.
So, well… From inside the community, the core team’s stance feels like pretty cautious/conservative, but I believe it might not seem so comparing to other communities.
It doesn’t seem anything special really. Of course Python 2 to 3 was a much bigger change (since they decided “oh, we are going to do breaking changes anyway, let’s fix all those small things that were bothering us for a while”), but at the tail end of the migration most of the hold ups were random scripts written by a Ph. D. trying to run some experiments. I think if anything, it does seem to me that big corporations were one of the biggest pushers for Python 3 once it became clear that Python 2 was going to go EOL.
I’d say that the keyword splatting nil change is probably not as breaking as the frozen string literal or even the it change (though I do not know the implementation details of the latter, so it might not be as breaking as I think). And for frozen string literals, they’ve been trying to make it happen for years now. It was scheduled to be the default in 3 and was put off for 4 whole years because they didn’t want to break existing code.
Over the years I feel like Ruby shops have been dedicated to keeping the code tidy and up-to-date. Every Ruby shop I’ve been at has had linting fail the build. Rubocop (probably the main linter now) is often coming out with rule adjustments, and often they have an autocorrect as well making it very easy to update the code. These days I just write the code and rubocop formats and maybe adjusts a few lines, I don’t mind.
I always think about the transition between Python 2 and 3 that the major change was adopting UTF-8 and everyone lost their minds thanks to the breakage, but Ruby did a similar migration in the version 2.0 and I don’t remember anyone complaining.
From what I remember, UTF-8 itself wasn’t the problem— most code was essentially compatible with it. The problem was that in Python 2 you marked unicode literals with u"a u prefix", and Python 3 made that a syntax error. This meant a lot of safe Python 2 code had to be made unsafe in Python 2 in order to run in Python 3. Python 3.3 added unicode literals just to make migrations possible.
On top of that, Python 3 had a lot of other breaking changes, like making print() a function and changing the type signatures of many of the list functions.
As someone who was maintaining a python package and had to make it compatible with 2 and 3, it was a nightmare. For instance the try/except syntax changed.
Python 2
try:
something
except ErrorClass, error:
pass
Python 3
try:
something
except ErrorClass as error:
pass
Basically the same thing but both are syntax error in the other version, that was a nightmare to handle. You can argue the version 3 is more consistent with other construct but it’s hard to believe it would have been particularly hard to support both syntax for a while to ease the transition.
Ruby change way more things, but try its best to support old and new code for a while to allow a smooth transition. It’s still work to keep up, but it’s smoothed out over time making it acceptable to most users.
It’s been a while, and I was just starting out with Python at the time, so take this with a grain of salt, but I think the problem was deeper than that. Python 2’s unicode handling worked differently to Python 3, so even when Python 3 added unicode literals, that didn’t solve the problem because the two string types would still behave differently enough that you’d run into compatibility issues. Certainly I remember reading lots of advice to just ignore the unicode literal prefix because it made things harder than before.
Googling a bit, I think this was because of encoding issues — in Python 2, you could just wrap things in unicode() and the right thing would probably happen, but in Python 3 you had to be more explicit about the encoding when using files and things. But it’s thankfully been a while since I needed to worry about any of this!
My recollection at Dropbox was that UTF8 was the problem and the solution was basically to use mypy everywhere so that the code could differentiate between utf8 vs nonutf8 strings.
In my experience the core issue was unicode strings and removing implicit encoding / decoding was, as well as updating a bunch of APIs to try and clean things up (not always successfully). This was full of runtime edge cases as it’s essentially all dynamic behaviour.
Properly doing external IO was on some concern but IME pretty minor.
On top of that, Python 3 had a lot of other breaking changes, like making print() a function and changing the type signatures of many of the list functions.
This is why I said the “major” change was UTF-8. I remember lots of changes were trivial (like making print a function, you could run 2to3 and it would mostly fix it except for a few corner cases).
To me, the big problem wasn’t so much to convert code from 2 to 3, but to make code run of both. So many of the “trivial” syntax changes were actually very challenging to make work on both versions with the same codebase.
It was a challenge early on, after ~3.3 it was mostly a question of having a few compatibility shims (some very cursed, e.g. if you used exec) and a bunch of lints to prevent incompatible constructs.
The string model change and APIs moving around both physically and semantically were the big ticket which kept lingering, and 2to3 (and later modernize/ futurize) did basically nothing to help there.
It wasn’t an easy transition. As others said, you’re referring to the 1.8-1.9 migration. It was a hard migration. It took around 6-7 years. An entirely.new VM was developed. It took several releases until.there was a safe 1.9 to migrate to, which was 1.9.3 . Before that, there were memory leaks, random segfaults, and one learned to avoid APIs which caused them. Because of this, a big chunk of the community didn’t even try 1.9 for years. It was going so poorly that github maintained a fork called “ruby enterprise edition”, 1.8 with a few GC enhancements.
In the end , the migration was successful. That’s because, once it stabilised, 1.9 was significantly faster than 1.8 , which offset the incompatibilities. That’s why python migration failed for so long: all work and no carrot. For years, python 3 was same order of performance or worse than python 2. That only changed around 3.5 or 3.6 .
Fwiw the ruby core team learned to never do that again, and ruby upgrades since 1.9 are fairly uneventful.
Ruby 2 was a serious pain for many large projects. Mainly with extensions behaving slightly differently with the encoding. I remember being stuck on custom builds of 1.9 for ages at work.
So merry ChristMatz 2022, we got 3.2 where YJIT is stable but off by default. 2023 it 3.3, YJIT’s on by default in a paused state so you can enable it in code, rather than when compiling and passing a flag to executables. No relevant change this year.
Anyone got a sense of when it’ll be on by default? Or remove the flag altogether? Why might someone avoid YJIT, and why can’t we give em an LTS version?
I dunno, seeing all that feature flagging in the low level implementation puts a squirm in my spine. I don’t think it’s like wrong, I just wouldn’t want to be maintaining it. OTOH, that’s all focused and prolly low churn stuff, so I suppose it ain’t hurting no one.
I believe another reason it is still off by default is more people-related: ruby-core is mostly Japanese and C-based. YJIT is maintained by Shopify employees and Rust-based.
Ruby has many of the same problems as Linux – some groups want to move to Rust for safety and speed but the existing maintainers often don’t know Rust at all.
Ruby has improved quite a lot. I love Ruby and used it extensively in the late 2000s. MRI was a disaster. Slow and had memory leak issues in long-running processes. Right now, it’s much faster and more stable. Night and day difference. Leaving Ruby aside, it never ceases to amaze me how good and performant the JVM is. The results in this benchmark are quite stunning.
yea, it’s a great question, and something I thought of mentioning in the article (then forgot).
I think the primary reason it’s still off by default is because of memory overhead. any JIT will usually add a not insignificant amount of memory usage on top of the runtime.
That said, YJIT is really configurable about the amount of memory it will consume. By default I think it’s 48 megs per process max? I know Github uses it but tunes it down a bit to more like 16megs. So possibly in the future it will be on, but by default set to a lower max.
Would be curious to hear from the YJIT team on their thoughts on that!
48MiB is how much executable memory it will use, but it also has to keep track of some metadata which usaly account for 2 or 3x the executable memory. So you’ll easily end up with a 150-200MB overhead with the default settings.
And yes you are right, the main blocker for enabling it by default really is extra memory usage, even though the YJIT team never formally proposed it, but from informal discussion with other Ruby committers, it’s clear that it would be the blocker.
I think the main reason to not flip it on by default in 3.3+ was that it could break container / memory-constrained environments if folks upgrade blindly and don’t monitor and increase mem limits appropriately It also can increase startup time for interactive use cases, such as CLIs and such.
I dunno if that was really the right call, but it seems that more conservative approach still holds: I haven’t heard any change in the default YJIT settings for 3.3+.
It’s interesting. I would think that you’d want everything written in Ruby (which would allow for JIT optimizations), and then hand pick things to optimize in C-Ruby (for interpretation), but they’re doing the exact opposite! :D
If they were starting from scratch, but with YJIT, I’m sure they would. They’re undoing earlier manual optimizations in favor of the new thing that’ll do it automagically.
In the past (10+ years ago) the primary path to improve ruby MRI perf was to optimize all the hot code paths via C. Or you could go alternative ruby runtimes with rubinius or jruby or one of the many others, but mri has always been the primary runtime. This pre-dated any sort of production-ready JIT development for MRI.
So now I think there a lot of perf sensitive code paths where you have to carefully unwind C (or essentially feature flag, as the post shows) in ruby-core to let YJIT do its magic.
IIRC Ruby 3.3 does not have YJIT on by default but 3.4 will. With that change they can modify the codebase to favor refactors that help JIT versus today they need to balance both on and off execution mode performance.
I maintain the main Redis client for Ruby, and similarly was contacted by Redis Inc a couple months back.
They asked about adding some features (client side caching, and support for various extensions they’re bringing in core), told them they could submit PRs like anyone else, haven’t seen anything from them yet.
They then asked about co-ownership of the package, I told them if they want it they could have it for themselves but that I wouldn’t co-maintain with someone I don’t know, they immediately backtracked, suggesting they don’t have the means to maintain it themselves.
My understanding is that they’re selling support and get various requests from their customers that require client side support. I don’t think they particularly want to own the clients, but if some key features are missing it’s a problem for them. From what I understand (might be wrong) redis-rs is somewhat in low maintenance mode, which may be what prompted them to try to take ownership.
All this to say, I don’t think they have a grand strategy to get ownership of all the clients, I think they only want clients to implement support for the features they add to the server, otherwise their offering is useless.
NB: I’m only guessing on their motivation, I’m not defending their use of trademark to get what they want.
So, you’re saying that there’s a company whose proprietary builds on open source without giving back? Have you considered changing the license on the Ruby redis client so that they can’t use it anymore? That seems to be the fashion these days 😉
As for changing the license, I don’t see what it will achieve except harm Ruby users. A Redis client can be used with actual Redis or plenty of alternatives including ValKey. If Redis Inc submit PRs to the Ruby client I’ll happily merge them if they make sense, to me they’re just like any other user/contributor.
It just seems that Redis Inc is doing the exact thing that they say motivated them to stop licensing Redis under an open source license. I don’t actually think you should do that (hence the wink-emoji), and I don’t think that Redis Inc should done it with the code they maintain either.
It’s just entirely predictable that these companies that claim that they’re being exploited by bigger players end up doing the same thing to smaller players.
I don’t feel exploited though. Maybe they tried to exploit me, but I’m not even sure of that.
I write OSS because I want to, and am perfectly aware people and companies can use that code and make money out of it. If I minded I wouldn’t release code under such licenses.
IMO the problem with Redis Inc isn’t that they’re a private company, it’s that they’ve raised lots of money and now need to justify their valuation, and found their 900 employees.
I don’t think the old, and much more “lean”, support contract model would have caused them to do that license change.
As far as I’m aware, there’s no FSF-, OSI-, or DFSG-approved license which would restrict usage on the opposite end of a socket.
The whole “once it’s in another process, it’s out of scope for this license” is an intentional feature, not a bug. (See “Running GOG.com games on the Linux kernel”, “GPLed archiver GUIs shelling out to unrar despite its GPL-incompatible license, being able to make a GPLed HTTP server/client and connect it to GPL-incompatible HTTP clients/servers, etc. etc. etc.)
(And people generally don’t want dependencies that the Debian and Fedora legal teams reject. It’s bad enough that Open Watcom C/C++ has that forced share-your-changes clause that keeps the best DOS/DPMI/Win16/NetWare/etc. toolchain for retro projects out of the repos by contravening debian-legal’s “desert island test”.)
They then asked about co-ownership of the package, I told them if they want it they could have it for themselves but that I wouldn’t co-maintain with someone I don’t know, they immediately backtracked, suggesting they don’t have the means to maintain it themselves.
To me, this is the most telling/annoying thing. They’re probably making money hand-over-fist providing consulting services for Redis, but yet don’t want to re-invest some of those profits to make the client libraries better. They want control, but don’t want to put in the effort.
I think one big difference from Rails to the “random web framework” is that Rails was extracted from an actual product (Basecamp) instead of being created as a web framework from scratch.
So DHH and others knew what worked for their business requirements, and you can see this even to this day: most of the Rails new features are created first from requirements from 37Signals’ (Basecamp creators) products.
That is not to say that every framework to be successful needs to be an actual product attached to it, but having it means Rails evolves accordingly to the necessities of 37Signals products (well, and other contributors like GitHub), instead of evolving due to the needs from alien codebases that you don’t have access. And I think this allows Rails to make controversial changes that would not be difficult to push in other Web frameworks.
Even more interesting, LJW has long since migrated to WordPress. 😭
I think the business problem was that news shrank and a random newspaper in Kansas could no longer afford to have a whole team of web developers. With WordPress, you just pay a consultant once every two years and have a semi-technical editor do the upkeep in between.
That’s interesting. I think a CMS is a product that always wants to do more and be configured in many ways. Maybe that’s the difference that basecamp was more focused and more like what the average web developer was likely to create or need.
That is indeed the case with CMS tools, but the economics don’t support in-house development outside of the biggest news organizations. In media, it’s common for publications to be bought and sold. The history in the LJW Wikipedia article is similar to many other publications. Each time it changes hands, there is an evaluation by the purchaser of what they are buying and how it’ll fit into their portfolio. At some point that development will have run out of support.
Contrast this situation with Basecamp. It has continuity of ownership and is ostensibly generating solid profits for 37signals to the point that they can staff a significant open-source project. It seems that the backing from 37signals is such that they could possibly get by on their own (EDIT: not that that approach is beneficial.)
Tobias Lütke is still on the core rails team i believe.
I’m a top 50 contributor and have attended almost every RailsConf and rubyconf since 2012. I’ve never met him in person or seen a commit by him during that time. Not saying it didn’t happen, but I would not call say he is “on the core team” or even close for many years. An alum, yes. But not active recently.
Also he called me a “human Godwin’s law” on Twitter for asking that he uphold the Shopify TOS and boot the Breitbart store.
Shopify is still driving some heavy duty improvements to the Ruby web story, although many of them are lower on in the stack recently. The LSP, JIT system(s), and Sorbet (the first big Ruby static typing system), off the top of my head.
Not just Ruby web story. They are a heavy contributor to the language overall. I can list so many of their contributions. Salient ones I can recall include YJIT, Ruby LSP etc…
The performance degradation when Python is built with free-threading support is significant – around 20%.
While Mandelbrot isn’t necessarily a super representative benchmark of what Python is generally used for, It’s nice to see what the perf degradation is like. I wonder how much they’ll be able to reduce it.
I hope this somehow manages to boost the velocity of nginx development, but I’m not optimistic at this point. After everything has happened with it, culminating with one of the core developers forking the project a while ago, I’m somewhat concerned about the project’s future under the current ownership structure.
Maybe it’s just me, but it looks like the development has slowed to a crawl and apart from CVE patches and minor, incremental feature updates and enhancements, there’s simply nothing exciting going on for nginx these days. Real shame, it’s a solid, battle-tested reverse and mail proxy and it pains me to see it almost abandoned by everybody.
Nothing wrong with that. However, there are some of us working at huge content delivery companies and the like and having a reverse proxy that keeps up to date with all the standards, protocols and new cool features deriving from new best practices/RFCs is something that’s very desirable.
For personal use cases, I agree, a reverse proxy should just work with minimal upkeep requirements.
If you’re a huge content delivery company that relies on nginx for your product but you haven’t hired an nginx developer to implement the features you want, then you might understand when I have a difficult time mustering up much sympathy for your plight.
Yes, you will hire nginx developers to develop custom functionality, but at one point, if the overall development on the upstream slows down to a crawl, then you either need to keep a private fork for yourself and maintain it indefinitely, in which case good luck finding new developers in the future, or you need to switch to something else (like many CDN companies are looking into Pingora and what to develop on top of it, for example) entirely.
I just happen to know these days it’s somewhat challenging to find proper nginx developers, so the “decay” is already a thing to a certain extent. With too many forks and core developers leaving, it doesn’t bode well for the project overall. I wish I’ll turn out to be wrong one day, though.
it sounds like most large companies have decided to stop investing in the public project, and have either built their own private nginx extensions or used a totally different code base. that’s their decision. as for smaller companies and individuals, it seems like the battle-tested nature of nginx outweighs the new standards and features that are mostly just useful for large companies anyway.
seeing it as a natural consequence of the divergence of interests between different groups of potential users, may help clarify what is disappointing or unsatisfying about the current state of the project and who can reasonably be expected to act to remedy the situation.
Yup. nginx has reached a certain level of maturity where it simply doesn’t need anything else in order to be the 1st choice for many use cases. But protocols, underlying technologies and new features are not standing still and it’s only a matter of time before nginx becomes irrelevant now that the pace of development has decelerated significantly with too many core developers forking the project and going their own way.
In one of my comments down below, I outlined the case of Cloudflare and their Pingora framework they open sourced a while ago. It didn’t happen just like that, and it’s an early indication of what’s in store for nginx should this situation continue for it.
Also, due to my career, I just happen to know that many big CDN players are considering Pingora for their next-gen platform and all of a sudden, a lot of them are looking into the cost-benefit of ditching nginx in the long term and going with their own in-house solution. Party like it’s 1999.
Can you elaborate on what exciting areas or features that you think are missing? I casually peeked at the broader nginx mailing lists, https://mailman.nginx.org/mailman/listinfo and see them to be reasonably active.
As for functionality, the open source version is pretty solid and just works. I love the simple configuration, and the feature set it provides (serves my purpose).
Well, QUIC+HTTP/3 support is still experimental, and that’s slowly becoming an issue for major deployments out there. Also, the configuration language, although extremely versatile, doesn’t support if/else scenarios (yes, I know if is evil in nginx, but I tend to know what I’m doing with it) for more advanced URI matching rules making me think of very creative ways of overcoming that situation, etc.
Need session stickiness for upstreams? Good luck finding a 3rd party module that’s still maintained or else be ready to either write it yourself or shell out the monies for the nginx Plus version that’s got that built in.
There are plenty more features that are simply missing and IMO should be a part of the modern open source reverse proxy solution in 2024.
Nothing per se, but in the case of nginx, the pricing structure has always been somewhat peculiar with a very steep pricing from the get go and eventually you run the risk of ending up in a typical vendor lock in situation while you’ve spent a ton of money along the way, at which point it becomes cheaper and overall better to make your own solution from scratch. Hint: CloudFlare Pingora - they never were a nginx Plus customer but they spent a ton of money and engineering hours trying to push the limits of nginx only to realize it’d be better for them to just make their own thing. We’re yet to see amazing things built on top of Pingora, reminds me of 2006-2009 period of nginx where every now and then something new and useful would pop out.
Is there any web server with a good HTTP/3 at the moment? My impression was that everyone was struggling with QUIC and there HTTP/3. AFAIU, there are working implementations of QUIC but the performance for end users isn’t necessarily better than TCP.
The performance of QUIC is a whole other story, however, it’s here to stay and it’s slowly starting to gain traction. There are many implementations for various web servers, none of them are getting it right at the moment, but the fact that QUIC support in nginx is still experimental in 2024 is a bit worrying to me.
why do you say that QUIC is here to stay and starting to gain traction if no web servers properly support it? why is it worrying for adoption to be slow if there’s no consensus that it’s an improvement?
QUIC is definitely here to stay because too many companies are investing more money into QUIC deployments and further R&D to make it better. One of the reasons why the adoption is slow is that it’s such a radical departure from the whole TCP/IP architecture and it turns out that network stacks and a bunch of other software was simply never that good in handling large UDP traffic so far, simply because there was no need to do so.
It’s going to take a while before everyone in the hierarchy gets things right, but based on what I see in the industry overall, we’re slowly starting to see some kind of resemblance of the light at the end of the tunnel. In fact, just today a very nice comment appeared on the site which sums some of the major challenges quite nicely.
I expect QUIC to become widely deployed but I was recently pondering having HTTP/3 on my servers and immediately stopped because I had no idea what to use. This makes me wonder if it’s ever going to replace HTTP/1 for most servers or if it will only be used by the largest players.
Huh, I didn’t realize HTTP/2 Push was so unpopular/underutilized. Intuitively, it makes sense – when you’re initially serving the index page for a SPA, you know the client is about to request your CSS and JS, so you might as well save a roundtrip. I guess it doesn’t make enough of a difference for most people to bother. But what do I know?
I guess it’s just really hard to implement. If you’re a web server, you would need to parse the HTML to know what resources would be fetched, and then what about resources fetched through JavaScript? That’d basically need the framework to push out the resources to the web server, but that wouldn’t work for e.g. static sites or CDNs (which are hosting ~all of the stuff you’d use HTTP/2 for). Turns out rel=“preload” combined with HTTP/3 works well enough without requiring web servers to implement an HTML parser.
Its latency wins are also a bit dubious because you may be forcing the retrieval of something the client already has in cache, and thus the various early hint mechanisms instead - the client knows its own cache contents.
In addition to requiring coupling between the pages/application and the web server (which might be a reverse proxy and therefore separate!), it wasn’t bringing much IIRC.
The alternative is having your browser parse the HTML it receives, start parsing it (which it does very well), spot resources it will require, and request them. Since the resources are often listed at the beginning of the HTML, the browser will do that quickly and often there isn’t really a gap.
Moreover, I think it could actually lead to a lower performance (too many elements listed, some unused, …)
hmm yeah maybe room for an integer id option. eg https://sqids.org/ruby or similar. i think in most cases i’d rather have a big index than two ids though. it’s nice being able to drop into psql and search by the same id that’s in a url.
I didn’t mean to have two ids, but to have a way to decode these string ids into 64 or even 128but integers so they are stored as such in the DB, like UUIDs basically
do you know if you could have it produce the non integer representation with a select * right in postgres somehow? or would that require passing it through a function (or maybe a virtual column or something like that)
It’s great to see Sidekiq still going strong after all these years! The iteration feature is really interesting, and I can think of many situations where it would have come in very handy but I had to reach for a more complicated solution. Don’t think I’ve seen that in similar libraries.
It’s directly adapted from https://github.com/Shopify/job-iteration
Every big Rails team I have worked on ends up building their own janky version of it (I’ve built a couple), very nice to have Sidekiq provide an implementation.
I’ve been longing for something like this.
Every place I’ve ever worked I’ve always had a machine capable of running the test suite magnitudes faster than CI. I used to jokingly tell my colleague “What if I could just tell GH the tests pass” while waiting for them to finish
My workflow is typically: run all the tests locally (very fast), push up code then wait an 10 extra minutes just to confirm the same thing (that all the tests pass).
If that’s the case, the CI hardware is really anemic, or there’s something really wrong with its setup.
So it sounds a bit like throwing the baby with the bathwater.
I think this is a great idea, but I am anticipating folks explainIng why it isn’t.
The main argument against is that even if you assume good intentions, it won’t be as close to production as an hosted CI (e.g. database version, OS type and version, etc).
Lots of developers develop on macOS and deploy on Linux, and there’s tons of subtle difference between the two systems, such as case sensitivity of the filesystem, as well as default ordering just to give an example.
To me the point of CI isn’t to ensure devs ran the test suite before merging. It’s to provide an environment that will catch as many things as possible that a local run wouldn’t be able to catch.
I’m basically repeating my other comment but I’m amped up about how much I dislike this idea, probably because it would tank my productivity, and this was too good as example to pass up: the point of CI isn’t (just) to ensure I ran the test suite before merging - although that’s part of it, because what if I forgot? The bigger point, though, is to run the test suite so that I don’t have to.
I have a very, very low threshold for what’s acceptably fast for a test suite. Probably 5-10 seconds or less. If it’s slower than that, I’m simply not going to run the entire thing locally, basically ever. I’m gonna run the tests I care about, and then I’m going to push my changes and let CI either trigger auto-merge, or tell me if there’s other tests I should have cared about (oops!). In the meantime, I’m fully context switched away not even thinking about that PR, because the work is being done for me.
You’re definitely correct here but I think there are plenty of applications where you can like… just trust the intersection between app and os/arch is gonna work.
But now that I think about it, this is such a GH-bound project and like… any such app small enough in scope or value for this to be worth using can just use the free Actions minutes. Doubt they’d go over.
Yes, that’s the biggest thing that doesn’t make sense to me.
I get the argument that hosted runners are quite weak compared to many developer machines, but if your test suite is small enough to be ran on a single machine, it can probably run about as fast if you parallelize your CI just a tiny bit.
I wonder if those differences are diminished if everything runs on Docker
With a fully containerized dev environment yes, that pretty much abolish the divergence in software configuration.
But there are more concern than just that. Does your app relies on some caches? Dependencies?
Where they in a clean state?
I know it’s a bit of an extreme example, but I spend a lot of time using
bundle openand editing my gems to debug stuff, it’s not rare I forget togem pristineafter an investigation.This can lead me to have tests that pass on my machine, and will never work elsewhere. There are millions of scenarios like this one.
I was once rejected from a job (partly) because the Dockerfile I wrote for my code assignment didn’t build on the assessor’s Apple Silicon Mac. I had developed and tested on my x86-64 Linux device. Considering how much server software is built with the same pair of configurations just with the roles switched around, I’d say they aren’t diminished enough.
Was just about to point this out. I’ve seen a lot of bugs in aarch64 Linux software that don’t exist in x86-64 Linux software. You can run a container built for a non-native architecture through Docker’s compatibility layer, but it’s a pretty noticeable performance hit.
One of the things that I like having a CI is the fact that it forces you to declare your dev environment programmatically. It means that you avoid the famous “works in my machine” issue because if tests works in your machine but not in CI, something is missing.
There are of course ways to avoid this issue, maybe if they enforced that all dev tests also run in a controlled environment (either via Docker or maybe something like testcontainers), but it needs more discipline.
This is by far the biggest plus side to CI. Missing external dependencies have bitten me before, but without CI, they’d bite me during deploy, rather than as a failed CI run. I’ve also run into issues specifically with native dependencies on Node, where it’d fetch the correct native dependency on my local machine, but fail to fetch it on CI, which likely means it would’ve failed in prod.
Here’s one: if you forget to check in a file, this won’t catch it.
It checks if the repo is not dirty, so it shouldn’t.
This is something “local CI” can check for. I’ve wanted this, so I added it to my build server tool (that normally runs on a remote machine) called ding. I’ll run something like “ding build make build” where “ding build” is the ci command, and “make build” is what it runs. It clones the current git repo into a temporary directory, and runs the command “make build” in it, sandboxed with bubblewrap.
The point still stands that you can forget to run the local CI.
What’s to stop me from lying and making the gh api calls manually?
Not a Ruby guy but I’ve found this series of posts to be fantastic.
Thanks!
This was a very Ruby-centric take (not surprising, the author is on the Rails core team). It’s true that if you’re not a CDN then HTTP/2 probably has little impact on you as a developer…unless you use bi-directional streaming.
HTTP/2 adds bi-directional streaming, which moves HTTP from a serial request-response model to a concurrent request-response model. This unlocks way richer API design in browser-less environments like backend services. It’s probably most often utilized in gRPC services that use client and/or server streaming.
As a longtime-sometimes Rubyist, I get why there’s not a lot of love for bi-directional streaming. But it’s disappointing because it means that there are better languages & frameworks out there for certain types of backend APIs.
Yes, everything published on my blog should be read from a Ruby point of view and I believe the intro does a good enough job at explaining the scope of my opinion:
And yes, gRPC is probably the only reason you’d want full HTTP/2 support in a Ruby server, but the official
grpcbindings for Ruby are so terrible that I wouldn’t recommend to try to do gRPC with Ruby to my worst enemy, and much simpler protocols like Twirp would give you 90% of the capabilities with much less headaches.So it’s not really among my concerns.
The wording of the blog post confused me, because in my mind “FFI” (Foreign Function Interface) usually means “whatever the language provide to call C code”, so in particular the comparison between “the function written as a C extension” and “the function written using the FFI” is confusing as they sound like they are talking about the exact same thing.
The author is talking specifically about a Ruby library called
FFI, where people write Ruby code that describe C functions and then the library is able to call them. I would guess that it interprets foreign calls, in the sense that the FFI library (maybe?) inspects the Ruby-side data describing their interface on each call, to run appropriate datatype-conversion logic before and after the call. I suppose that JIT-ing is meant to remove this interpretation overhead – which is probably costly for very fast functions, but not that noticeable for longer-running C functions.Details about this would have helped me follow the blog post, and probably other people unfamiliar with the specific Ruby library called
FFI.Replying to myself: I wonder why the author needs to generate assembly code for this. I would assume that it should be possible, on the first call to this function, to output C code (using the usual Ruby runtime libraries that people use for C extensions) to a file, call the C compiler to produce a dynamic library,
dlopenthe result, and then (on this first call and all future calls) just call the library code. This would probably get similar performance benefits and be much more portable.I would guess because that would require shipping a C compiler? Ruby FFI/C extensions are compiled before runtime; the only thing you need to ship to prod is your code, Ruby, and the build artifacts.
This is essentially how MJIT worked.
https://www.ruby-lang.org/en/news/2018/12/06/ruby-2-6-0-rc1-released/ https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch#mjit-organization
Ruby has since that evolved very fast on the JIT side, spawning YJIT, RJIT, now FJIT…
I’m also not sure if the portability is needed here. Ruby is predominantly done on x86, at least at the scale where these optimisations matter.
Apple Silicon exists and is quite popular for development
You’re correct. I was referring to deployment systems (where the last bit of performance matters) and should have been clearer about that.
Even in production, ARM64 is getting more common these days, because of AWS Graviton and al.
But yes, x86_64 is still the overwhelming majority of production deployments.
Yeah, that’s why I wrote “predominantly”. Also, for such a localised JIT, a second port to aarch64 is not that hard. You just won’t have an eBPF port falling out of your compiler (this is absurd for effect, I know this isn’t a reasonable thing).
Note that the point here is not to JIT arbirary Ruby code, which is probably quite hard, but a “tiny JIT” to use compilation rather than interpretation for the FFI wrappers around external calls. (In fact it’s probably feasible, if a bit less convenient, to set things up to compile these wrappers ahead-of-time.)
Seems like a bad idea to me. It’s too easy to have code that works until it doesn’t because the provided data stopped adding the extra null byte. If it’s really valuable to save that one byte, then it probably should be a different string
TYPE_.Also this is simpler than messagepack, but not that much simpler. The
TYPE_STRINGREFis a nice touch though, particularly for the classic “list of objects” JSON structure.It doesn’t seem complicated. Use SQLite for your single user desktop application, use PostgreSQL for a multi-user server, and don’t expect that the default configuration is well suited for your machine or your workload. That last is true for any data store (including filesystems, not just databases).
This was true, at least for ruby on rails about one year ago. But recently rails developers merged some PRs to leverage the concurrency features of the sqlite database. So, this may no longer be true.
SQLite non-GVL-blocking, fair retry interval busy handler
Ensure SQLite transaction default to IMMEDIATE mode
SQLite on Rails: Supercharging the One-Person Framework - Rails World 2024
SQLite on Rails: The how and why of optimal performance
Maybe @byroot can shed more light on this issue.
I personally think the above advice is still sound. I merged these SQLite improvements to Rails, because why not, but I’m personally very skeptical and wary of the current SQLite hype (if not over-hype).
If you are dead certain you’ll never need to span your application on more than one server, SQLite may make sense, but even then it might not be the most efficient solution.
I see how not having to administrate a PostgeSQL or MySQL service is attractive, but you’ll still have to sort out backups and such. You can’t just copy the files you have to lock the DB while doing so etc.
Also even with WAL etc, SQLite performs very well with reads, but not so well if your application does frequent writes, and with Ruby’s GVL it can happen that you cause the write lock to be held for much longer than you’d want.
So there’s some use cases where SQLite makes perfect sense (single machine, read heavy apps), but to me it’s the exception not the rule. I’d definitely consider SQLite if I had a use case that call for it, but I’d never make it part of my default stack (as in the stack I use when requirements aren’t super clear yet).
Regarding backups, sqlite now comes with tooling to make this easier (the Online Backup API and this tool: https://www.sqlite.org/rsync.html).
My comment had nothing to do with Rails. It’s due to differences in the underlying databases.
This is a very interesting article. I was originally taken aback by the initial “Not IO bound” comment, but pointing out that our current understanding of IO is actually conflated with the OS scheduling of threads was very on point. I hadn’t considered that before. I think my reaction still stands though, but in a pedantic way. Looking at:
and
I still feel that if a component sees a 2-3x perf increase and that translates to (1.15x-1.27x) improvement that it’s a significant component (and well worth optimizing), but it isn’t the dominant/limiting factor.
Towards the end of the article Jean gets into some specific numbers regarding “truly IO bound” being 95% and “kinda” being 50%. I asked him on Mastodon about them. https://ruby.social/@byroot/113877928374636091. I guess in my head “more than 50%” would be what I would classify as “IO bound.” Though I’ve never put a number to it before.
Someone tagged an old thread of mine in a private slack recently where I linked to this resource https://www.youtube.com/watch?app=desktop&v=r-TLSBdHe1A. With this comment
It would be interesting to have some kind of an IO backend where you could simulate a slowdown. I.e. perform the query on the database and time it, then sleep for some multiplier of that time before returning. It would (in theory) let you put a number to how much your app is affected by (database) IO. If you set a 2x multiplier and you see requests take 2x as long…then you’re approaching 100% IO bound.
The linked GVL timing gem is new and interesting. Overall, thanks for writing all this down. Great stuff.
Perhaps I should have written about that in my conclusion, but ultimately IO-bound isn’t perfect to describe all I want to say.
In a way it’s a good term, because the implication of an IO-bound app, is that the only way to improve its performance is to somehow parallelize the IOs it does, or to speedup the underlying system it does IOs with.
With that strict definition, I think YJIT proved that it isn’t the case, given it proved it was able to substantially speedup applications.
A more relaxed way I tend to use the IO-bound definition, in the context of Ruby applications, is whether or not you can substantially increase your application throughput without degrading its latency by using a concurrent server (typically Puma, but also Falcon, etc).
That’s where the 50% mark is important. 50% IO implies 50% CPU, and one Ruby process can only accommodate “100%” CPU usage. And given threads won’t perfectly line up when they need the CPU, you need substantially more than 50% IO if you wish to process concurrent requests in threads without impacting latency because of GVL contention.
So beyond trying say whether apps are IO-bound or not, I mostly want to explain under which conditions it makes sense to use threads or fibers, and how many.
Are you comparing multiple single threaded processes to a single multithreaded process here? Otherwise, I don’t understand your logic.
If a request takes 500msec of cpu time and 500msec of “genuine” io time, then 1 request per second is 100% utilization for a single threaded server, and queue lengths will grow arbitrarily. With two threads, the CPU is only at 50% utilization, and queue lengths should stay low. You’re correct that there will be some loss due to requests overlapping, and competing for CPU time, but it’ll be dominated by the much lower overall utilization.
In the above paragraph, genuine means “actually waiting on the network”, to exclude time spent on CPU handling the networking stack/deserializing data.
P.S. Not expressing an opinion on “IO bound” it’s not one of my favorite terms.
Yes. Usually when you go with single threaded servers like Unicorn (or even just Puma configured with a single thread per process), you still account for some IO wait time by spawning a bit more processes than you have CPU core. Often it’s 1.3 or 1.5 times as much.
I don’t think there’s anything special about 50% CPU. The more CPU time the worse, but I don’t think anything changes significantly at that point, I think it’s going to be relatively linear relationship between 40% and 60%.
You’re going to experience some slowdown (relative to multiple single-threaded processes), as long as the arrival rate is high enough that there are ever overlapping requests to the multithreaded process. Even if CPU is only 1/4 of the time, your expectation is a 25% slowdown for two requests that arrive simultaneously.
I think, but am not sure, that the expected slowdown is “percent cpu time * expected queue length (for the cpu)”. If queue length is zero, then no slowdown.
You can achieve some of that with a combination of cgroups limiting the database performance and https://github.com/Shopify/toxiproxy for extra latency/slowdown on TCP connections.
I got involved in a funny little conversation about how Lobsters’ performance informs this post. (backstory)
We should soon know if my expectations hold true for lobste.rs: https://github.com/lobsters/lobsters/pull/1442
If anyone’s curious, this is now live in prod. I’ll remove it roughly tomorrow morning (depends on my social calendar). I pasted a few lines from the log to the PR so it’s obvious there’s no personal info going to byroot.
I know I’ve said this before, but one of the reasons I find maintaining the Lobsters codebase rewarding is that it’s landed in a sweet spot where it handles real-world complexity and some scale, but is small, standard-style, and with few enough dependencies that devs can find their way around quickly. People have used it for corporate training on writing good tests, to learn or experiment with Rails, to improve Ruby itself, to design a database, to crib small examples from. We’ve benefited hugely from the generosity of the projects we’re built on and we get to share onwards in turn.
I’m excited to see the results!
Author here, happy to answer questions if any.
The blog series has been really interesting! Thank you both for the work making JSON better and writing about it.
One thing that I found interesting is that many changes in each Ruby release would be considered a big no for any other language (for example the “Keyword splatting nil” change) since the possibility of breaking existing code is huge, but Ruby community seems to just embrace those changes.
I always think about the transition between Python 2 and 3 that the major change was adopting UTF-8 and everyone lost their minds thanks to the breakage, but Ruby did a similar migration in the version 2.0 and I don’t remember anyone complaining.
I am not sure if this is just because the community is smaller, if the developers of Ruby are just better in deprecating features, or something else. But I still find interesting.
Ruby’s version of the Python 2 to 3 experience (by my memory) came years earlier, going from 1.8 to 1.9. It certainly still wasn’t as big of an issue as Python’s long-lingering legacy version, but it was (again, my perception at the time) the Ruby version that had the most lag in adoption.
Yes, and it was very well managed. For example, some changes were deliberately chosen in a way that you had to take care, but you could relatively easy write Ruby 1.8/1.9 code that worked on both systems.
The other part is that Ruby 1.8 got a final release that implemented as much as the stdlib of 1.9 as possible. Other breaking things, like the default file encoding and so on where gradually introduced. A new Ruby version is always some work, but not too terrible. It was always very user centric.
It was still a chore, but the MRI team was pretty active at making it less of a chore and getting important community members on board to spread knowledge and calm the waves.
Honestly, I think Ruby is not getting enough cred for its change management. I wish Python had learned from it, the mess of 2 vs 3 could have been averted.
Yep, that’s my take too. IIRC 1.9 had a number of breaking API changes which were really low value. For instance, File.exists? -> File.exist?
File.exists? started emitting deprecation warnings in Ruby 2.1 (2013) and was finally removed in Ruby 3.2 (2022)
I guess IDRC!
I feel like Python was pretty deeply ingrained in a bunch of operating systems and scripts that was excruciating to update.
Ruby is mostly run as web apps
Interesting POV. As a long-time Rubyist, I’ve often felt that Ruby-core was too concerned with backwards compatibility. For instance, I would have preferred a more aggressive attempt to minimize the C extension API in order to make more performance improvements via JIT. I’m happy to see them move down the path of frozen strings by default.
Like others already said, the Ruby core team stance is almost exactly the opposite: it is extremely concerned with backward compatibility and not breaking existing code (to the extent that during discussion of many changes, some of the core team members run
grepthrough the codebase of all existing gems to confirm or refute an assumption of the required change scale).As an example, the string literal freezing was discussed for many years, attempted before Ruby 3.0, was considered too big a change (despite the major version change); only pragma for opt-in was introduced, and now the deprecation is introduced in the assumption that the existence of pragma prepared most of the codebases for the future changes. This assumption was recently challenged, though, and the discussion is still ongoing.
Keyword splatting nil change might break only the code that relies on the impossibility of the
nilsplatting, which is quite a stretch (and the one that is considered acceptable in order to make any progress).This seems like really easy code to write and accidentally rely on.
If nil was expected but was just rolled up into the general error handling, this code feels very easy to write.
Well… it is relatively easy to write, yes, but in practice, this exact approach (blanket error catching as a normal flow instead of checking the argument) is relatively rare—and would rather be a part of an “unhappy” path, i.e., “something is broken here anyway” :)
But I see the point from which this change might be considered too brazen. It had never come out during the discussion of the feature. (And it was done in the most localized way: instead of defining
nil.to_hash—which might’ve been behaving unexpectedly in some other contexts—it is just a support for**nilon its own.)I have to doubt that. It’s extremely common in Python, for example, to catch ‘Exception’ and I know myself when writing Ruby I’ve caught
StandardError.I don’t have strong opinions.
I don’t mean catching
StandardErroris rare, I mean the whole combination of circumstances that will lead to “nilwas frequently splatted there and caught byrescue, and now it is not raising, and the resulting code is not producing an exception that would be caught byrescueanyway, but is broken in a different way”.But we’ll see.
But this doesn’t really matter, because there are always huge proprietary codebases that are affected for every change and you can’t run grep on them for obvious purposes. And those are the people that generally complain the most about those breaking changes.
Well, it matters in a way that the set of code from all existing gems covers a high doze of possible approaches and views on how Ruby code might be written. Though, of course, it doesn’t exclude some “fringe” approaches that never see the light outside the corporation dangeons.
So, well… From inside the community, the core team’s stance feels like pretty cautious/conservative, but I believe it might not seem so comparing to other communities.
It doesn’t seem anything special really. Of course Python 2 to 3 was a much bigger change (since they decided “oh, we are going to do breaking changes anyway, let’s fix all those small things that were bothering us for a while”), but at the tail end of the migration most of the hold ups were random scripts written by a Ph. D. trying to run some experiments. I think if anything, it does seem to me that big corporations were one of the biggest pushers for Python 3 once it became clear that Python 2 was going to go EOL.
I’d say that the keyword splatting nil change is probably not as breaking as the frozen string literal or even the
itchange (though I do not know the implementation details of the latter, so it might not be as breaking as I think). And for frozen string literals, they’ve been trying to make it happen for years now. It was scheduled to be the default in 3 and was put off for 4 whole years because they didn’t want to break existing code.Over the years I feel like Ruby shops have been dedicated to keeping the code tidy and up-to-date. Every Ruby shop I’ve been at has had linting fail the build. Rubocop (probably the main linter now) is often coming out with rule adjustments, and often they have an autocorrect as well making it very easy to update the code. These days I just write the code and rubocop formats and maybe adjusts a few lines, I don’t mind.
From what I remember, UTF-8 itself wasn’t the problem— most code was essentially compatible with it. The problem was that in Python 2 you marked unicode literals with
u"a u prefix", and Python 3 made that a syntax error. This meant a lot of safe Python 2 code had to be made unsafe in Python 2 in order to run in Python 3. Python 3.3 added unicode literals just to make migrations possible.On top of that, Python 3 had a lot of other breaking changes, like making
print()a function and changing the type signatures of many of the list functions.As someone who was maintaining a python package and had to make it compatible with 2 and 3, it was a nightmare. For instance the
try/exceptsyntax changed.Python 2
Python 3
Basically the same thing but both are syntax error in the other version, that was a nightmare to handle. You can argue the version 3 is more consistent with other construct but it’s hard to believe it would have been particularly hard to support both syntax for a while to ease the transition.
Ruby change way more things, but try its best to support old and new code for a while to allow a smooth transition. It’s still work to keep up, but it’s smoothed out over time making it acceptable to most users.
It’s been a while, and I was just starting out with Python at the time, so take this with a grain of salt, but I think the problem was deeper than that. Python 2’s unicode handling worked differently to Python 3, so even when Python 3 added unicode literals, that didn’t solve the problem because the two string types would still behave differently enough that you’d run into compatibility issues. Certainly I remember reading lots of advice to just ignore the unicode literal prefix because it made things harder than before.
Googling a bit, I think this was because of encoding issues — in Python 2, you could just wrap things in
unicode()and the right thing would probably happen, but in Python 3 you had to be more explicit about the encoding when using files and things. But it’s thankfully been a while since I needed to worry about any of this!My recollection at Dropbox was that UTF8 was the problem and the solution was basically to use mypy everywhere so that the code could differentiate between utf8 vs nonutf8 strings.
In my experience the core issue was unicode strings and removing implicit encoding / decoding was, as well as updating a bunch of APIs to try and clean things up (not always successfully). This was full of runtime edge cases as it’s essentially all dynamic behaviour.
Properly doing external IO was on some concern but IME pretty minor.
This is why I said the “major” change was UTF-8. I remember lots of changes were trivial (like making print a function, you could run
2to3and it would mostly fix it except for a few corner cases).To me, the big problem wasn’t so much to convert code from 2 to 3, but to make code run of both. So many of the “trivial” syntax changes were actually very challenging to make work on both versions with the same codebase.
It was a challenge early on, after ~3.3 it was mostly a question of having a few compatibility shims (some very cursed, e.g. if you used exec) and a bunch of lints to prevent incompatible constructs.
The string model change and APIs moving around both physically and semantically were the big ticket which kept lingering, and 2to3 (and later modernize/ futurize) did basically nothing to help there.
It wasn’t an easy transition. As others said, you’re referring to the 1.8-1.9 migration. It was a hard migration. It took around 6-7 years. An entirely.new VM was developed. It took several releases until.there was a safe 1.9 to migrate to, which was 1.9.3 . Before that, there were memory leaks, random segfaults, and one learned to avoid APIs which caused them. Because of this, a big chunk of the community didn’t even try 1.9 for years. It was going so poorly that github maintained a fork called “ruby enterprise edition”, 1.8 with a few GC enhancements.
In the end , the migration was successful. That’s because, once it stabilised, 1.9 was significantly faster than 1.8 , which offset the incompatibilities. That’s why python migration failed for so long: all work and no carrot. For years, python 3 was same order of performance or worse than python 2. That only changed around 3.5 or 3.6 .
Fwiw the ruby core team learned to never do that again, and ruby upgrades since 1.9 are fairly uneventful.
Minor correction: Ruby Enterprise Edition was maintained by Phusion (who did Passenger), not GitHub.
Ruby 2 was a serious pain for many large projects. Mainly with extensions behaving slightly differently with the encoding. I remember being stuck on custom builds of 1.9 for ages at work.
So merry ChristMatz 2022, we got 3.2 where YJIT is stable but off by default. 2023 it 3.3, YJIT’s on by default in a paused state so you can enable it in code, rather than when compiling and passing a flag to executables. No relevant change this year.
Anyone got a sense of when it’ll be on by default? Or remove the flag altogether? Why might someone avoid YJIT, and why can’t we give em an LTS version?
I dunno, seeing all that feature flagging in the low level implementation puts a squirm in my spine. I don’t think it’s like wrong, I just wouldn’t want to be maintaining it. OTOH, that’s all focused and prolly low churn stuff, so I suppose it ain’t hurting no one.
I believe another reason it is still off by default is more people-related: ruby-core is mostly Japanese and C-based. YJIT is maintained by Shopify employees and Rust-based.
Ruby has many of the same problems as Linux – some groups want to move to Rust for safety and speed but the existing maintainers often don’t know Rust at all.
Ruby has improved quite a lot. I love Ruby and used it extensively in the late 2000s. MRI was a disaster. Slow and had memory leak issues in long-running processes. Right now, it’s much faster and more stable. Night and day difference. Leaving Ruby aside, it never ceases to amaze me how good and performant the JVM is. The results in this benchmark are quite stunning.
yea, it’s a great question, and something I thought of mentioning in the article (then forgot).
I think the primary reason it’s still off by default is because of memory overhead. any JIT will usually add a not insignificant amount of memory usage on top of the runtime.
That said, YJIT is really configurable about the amount of memory it will consume. By default I think it’s 48 megs per process max? I know Github uses it but tunes it down a bit to more like 16megs. So possibly in the future it will be on, but by default set to a lower max.
Would be curious to hear from the YJIT team on their thoughts on that!
48MiB is how much executable memory it will use, but it also has to keep track of some metadata which usaly account for 2 or 3x the executable memory. So you’ll easily end up with a 150-200MB overhead with the default settings.
3.4 will have a much more ergonomic memory setting: https://github.com/ruby/ruby/pull/11810
And yes you are right, the main blocker for enabling it by default really is extra memory usage, even though the YJIT team never formally proposed it, but from informal discussion with other Ruby committers, it’s clear that it would be the blocker.
Ah right, thanks for the clarification byroot. Not the first time I’ve thought through that incorrectly - glad to have a clearer setting. Thanks!
I think the main reason to not flip it on by default in 3.3+ was that it could break container / memory-constrained environments if folks upgrade blindly and don’t monitor and increase mem limits appropriately It also can increase startup time for interactive use cases, such as CLIs and such.
I dunno if that was really the right call, but it seems that more conservative approach still holds: I haven’t heard any change in the default YJIT settings for 3.3+.
It’s interesting. I would think that you’d want everything written in Ruby (which would allow for JIT optimizations), and then hand pick things to optimize in C-Ruby (for interpretation), but they’re doing the exact opposite! :D
If they were starting from scratch, but with YJIT, I’m sure they would. They’re undoing earlier manual optimizations in favor of the new thing that’ll do it automagically.
In the past (10+ years ago) the primary path to improve ruby MRI perf was to optimize all the hot code paths via C. Or you could go alternative ruby runtimes with rubinius or jruby or one of the many others, but mri has always been the primary runtime. This pre-dated any sort of production-ready JIT development for MRI.
So now I think there a lot of perf sensitive code paths where you have to carefully unwind C (or essentially feature flag, as the post shows) in ruby-core to let YJIT do its magic.
IIRC Ruby 3.3 does not have YJIT on by default but 3.4 will. With that change they can modify the codebase to favor refactors that help JIT versus today they need to balance both on and off execution mode performance.
No, YJIT still isn’t on by default in 3.4.
I maintain the main Redis client for Ruby, and similarly was contacted by Redis Inc a couple months back.
They asked about adding some features (client side caching, and support for various extensions they’re bringing in core), told them they could submit PRs like anyone else, haven’t seen anything from them yet.
They then asked about co-ownership of the package, I told them if they want it they could have it for themselves but that I wouldn’t co-maintain with someone I don’t know, they immediately backtracked, suggesting they don’t have the means to maintain it themselves.
My understanding is that they’re selling support and get various requests from their customers that require client side support. I don’t think they particularly want to own the clients, but if some key features are missing it’s a problem for them. From what I understand (might be wrong)
redis-rsis somewhat in low maintenance mode, which may be what prompted them to try to take ownership.All this to say, I don’t think they have a grand strategy to get ownership of all the clients, I think they only want clients to implement support for the features they add to the server, otherwise their offering is useless.
NB: I’m only guessing on their motivation, I’m not defending their use of trademark to get what they want.
So, you’re saying that there’s a company whose proprietary builds on open source without giving back? Have you considered changing the license on the Ruby redis client so that they can’t use it anymore? That seems to be the fashion these days 😉
I don’t think I’ve said that.
As for changing the license, I don’t see what it will achieve except harm Ruby users. A Redis client can be used with actual Redis or plenty of alternatives including ValKey. If Redis Inc submit PRs to the Ruby client I’ll happily merge them if they make sense, to me they’re just like any other user/contributor.
It just seems that Redis Inc is doing the exact thing that they say motivated them to stop licensing Redis under an open source license. I don’t actually think you should do that (hence the wink-emoji), and I don’t think that Redis Inc should done it with the code they maintain either.
It’s just entirely predictable that these companies that claim that they’re being exploited by bigger players end up doing the same thing to smaller players.
I don’t feel exploited though. Maybe they tried to exploit me, but I’m not even sure of that.
I write OSS because I want to, and am perfectly aware people and companies can use that code and make money out of it. If I minded I wouldn’t release code under such licenses.
IMO the problem with Redis Inc isn’t that they’re a private company, it’s that they’ve raised lots of money and now need to justify their valuation, and found their 900 employees.
I don’t think the old, and much more “lean”, support contract model would have caused them to do that license change.
As far as I’m aware, there’s no FSF-, OSI-, or DFSG-approved license which would restrict usage on the opposite end of a socket.
The whole “once it’s in another process, it’s out of scope for this license” is an intentional feature, not a bug. (See “Running GOG.com games on the Linux kernel”, “GPLed archiver GUIs shelling out to
unrardespite its GPL-incompatible license, being able to make a GPLed HTTP server/client and connect it to GPL-incompatible HTTP clients/servers, etc. etc. etc.)(And people generally don’t want dependencies that the Debian and Fedora legal teams reject. It’s bad enough that Open Watcom C/C++ has that forced share-your-changes clause that keeps the best DOS/DPMI/Win16/NetWare/etc. toolchain for retro projects out of the repos by contravening
debian-legal’s “desert island test”.)I left the wink-emoji because yes, there’s no such open source license. But that’s the game that Redis Inc has been playing with this community.
Ahh. I vote to amend English grammar to add brace-scoping for emoji modifiers. :P
Bah, English grammar is an oxymoron!
English orthography and vocabulary are oxymorons. English grammar survived the Norman conquest and the Great Vowel Shift in pretty good shape.
That’s why it’s so well-suited to humour based around it, such as “Off is the direction in which I wish you to f**k” or “Superdickery”.
To me, this is the most telling/annoying thing. They’re probably making money hand-over-fist providing consulting services for Redis, but yet don’t want to re-invest some of those profits to make the client libraries better. They want control, but don’t want to put in the effort.
Actually I suspect they’re not making that much, hence their recent moves.
I think one big difference from Rails to the “random web framework” is that Rails was extracted from an actual product (Basecamp) instead of being created as a web framework from scratch.
So DHH and others knew what worked for their business requirements, and you can see this even to this day: most of the Rails new features are created first from requirements from 37Signals’ (Basecamp creators) products.
That is not to say that every framework to be successful needs to be an actual product attached to it, but having it means Rails evolves accordingly to the necessities of 37Signals products (well, and other contributors like GitHub), instead of evolving due to the needs from alien codebases that you don’t have access. And I think this allows Rails to make controversial changes that would not be difficult to push in other Web frameworks.
Interestingly, Django was also extracted from an actual product - the in-house CMS of the Lawrence (Kansas) Journal-World.
Even more interesting, LJW has long since migrated to WordPress. 😭
I think the business problem was that news shrank and a random newspaper in Kansas could no longer afford to have a whole team of web developers. With WordPress, you just pay a consultant once every two years and have a semi-technical editor do the upkeep in between.
That’s interesting. I think a CMS is a product that always wants to do more and be configured in many ways. Maybe that’s the difference that basecamp was more focused and more like what the average web developer was likely to create or need.
That is indeed the case with CMS tools, but the economics don’t support in-house development outside of the biggest news organizations. In media, it’s common for publications to be bought and sold. The history in the LJW Wikipedia article is similar to many other publications. Each time it changes hands, there is an evaluation by the purchaser of what they are buying and how it’ll fit into their portfolio. At some point that development will have run out of support.
Contrast this situation with Basecamp. It has continuity of ownership and is ostensibly generating solid profits for 37signals to the point that they can staff a significant open-source project. It seems that the backing from 37signals is such that they could possibly get by on their own (EDIT: not that that approach is beneficial.)
and shopify in the early days heavily influenced rails too. Tobias Lütke is still on the core rails team i believe.
I’m a top 50 contributor and have attended almost every RailsConf and rubyconf since 2012. I’ve never met him in person or seen a commit by him during that time. Not saying it didn’t happen, but I would not call say he is “on the core team” or even close for many years. An alum, yes. But not active recently.
Also he called me a “human Godwin’s law” on Twitter for asking that he uphold the Shopify TOS and boot the Breitbart store.
Achievement unlocked ♥
Shopify is still driving some heavy duty improvements to the Ruby web story, although many of them are lower on in the stack recently. The LSP, JIT system(s), and Sorbet (the first big Ruby static typing system), off the top of my head.
While Shopify is a relatively heavy user of Sorbet, and we do contribute to it frequently, we didn’t create it. It was done by Stripe.
As for being lower in the stack, it’s in part true, but we also release a lot of higher level tools, e.g. https://github.com/Shopify/maintenance_tasks to only quote one.
Not just Ruby web story. They are a heavy contributor to the language overall. I can list so many of their contributions. Salient ones I can recall include YJIT, Ruby LSP etc…
Tobi was a core contributor long time ago. Here is the current team: https://rubyonrails.org/community
lol, Tobi hasn’t been a dev in well over a decade, if he ever was
While Mandelbrot isn’t necessarily a super representative benchmark of what Python is generally used for, It’s nice to see what the perf degradation is like. I wonder how much they’ll be able to reduce it.
I hope this somehow manages to boost the velocity of nginx development, but I’m not optimistic at this point. After everything has happened with it, culminating with one of the core developers forking the project a while ago, I’m somewhat concerned about the project’s future under the current ownership structure.
Maybe it’s just me, but it looks like the development has slowed to a crawl and apart from CVE patches and minor, incremental feature updates and enhancements, there’s simply nothing exciting going on for nginx these days. Real shame, it’s a solid, battle-tested reverse and mail proxy and it pains me to see it almost abandoned by everybody.
I don’t know about you but “exciting” isn’t something I’m looking for in an HTTP server personally.
Nothing wrong with that. However, there are some of us working at huge content delivery companies and the like and having a reverse proxy that keeps up to date with all the standards, protocols and new cool features deriving from new best practices/RFCs is something that’s very desirable.
For personal use cases, I agree, a reverse proxy should just work with minimal upkeep requirements.
If you’re a huge content delivery company that relies on nginx for your product but you haven’t hired an nginx developer to implement the features you want, then you might understand when I have a difficult time mustering up much sympathy for your plight.
Yes, you will hire nginx developers to develop custom functionality, but at one point, if the overall development on the upstream slows down to a crawl, then you either need to keep a private fork for yourself and maintain it indefinitely, in which case good luck finding new developers in the future, or you need to switch to something else (like many CDN companies are looking into Pingora and what to develop on top of it, for example) entirely.
I just happen to know these days it’s somewhat challenging to find proper nginx developers, so the “decay” is already a thing to a certain extent. With too many forks and core developers leaving, it doesn’t bode well for the project overall. I wish I’ll turn out to be wrong one day, though.
it sounds like most large companies have decided to stop investing in the public project, and have either built their own private nginx extensions or used a totally different code base. that’s their decision. as for smaller companies and individuals, it seems like the battle-tested nature of nginx outweighs the new standards and features that are mostly just useful for large companies anyway.
seeing it as a natural consequence of the divergence of interests between different groups of potential users, may help clarify what is disappointing or unsatisfying about the current state of the project and who can reasonably be expected to act to remedy the situation.
Yup. nginx has reached a certain level of maturity where it simply doesn’t need anything else in order to be the 1st choice for many use cases. But protocols, underlying technologies and new features are not standing still and it’s only a matter of time before nginx becomes irrelevant now that the pace of development has decelerated significantly with too many core developers forking the project and going their own way.
In one of my comments down below, I outlined the case of Cloudflare and their Pingora framework they open sourced a while ago. It didn’t happen just like that, and it’s an early indication of what’s in store for nginx should this situation continue for it.
Also, due to my career, I just happen to know that many big CDN players are considering Pingora for their next-gen platform and all of a sudden, a lot of them are looking into the cost-benefit of ditching nginx in the long term and going with their own in-house solution. Party like it’s 1999.
this is the best-case trajectory for almost all projects IMO (minus the move to github)
Can you elaborate on what exciting areas or features that you think are missing? I casually peeked at the broader nginx mailing lists, https://mailman.nginx.org/mailman/listinfo and see them to be reasonably active.
As for functionality, the open source version is pretty solid and just works. I love the simple configuration, and the feature set it provides (serves my purpose).
Well, QUIC+HTTP/3 support is still experimental, and that’s slowly becoming an issue for major deployments out there. Also, the configuration language, although extremely versatile, doesn’t support if/else scenarios (yes, I know if is evil in nginx, but I tend to know what I’m doing with it) for more advanced URI matching rules making me think of very creative ways of overcoming that situation, etc.
Need session stickiness for upstreams? Good luck finding a 3rd party module that’s still maintained or else be ready to either write it yourself or shell out the monies for the nginx Plus version that’s got that built in.
There are plenty more features that are simply missing and IMO should be a part of the modern open source reverse proxy solution in 2024.
What’s wrong with shelling money?
Nothing per se, but in the case of nginx, the pricing structure has always been somewhat peculiar with a very steep pricing from the get go and eventually you run the risk of ending up in a typical vendor lock in situation while you’ve spent a ton of money along the way, at which point it becomes cheaper and overall better to make your own solution from scratch. Hint: CloudFlare Pingora - they never were a nginx Plus customer but they spent a ton of money and engineering hours trying to push the limits of nginx only to realize it’d be better for them to just make their own thing. We’re yet to see amazing things built on top of Pingora, reminds me of 2006-2009 period of nginx where every now and then something new and useful would pop out.
Is there any web server with a good HTTP/3 at the moment? My impression was that everyone was struggling with QUIC and there HTTP/3. AFAIU, there are working implementations of QUIC but the performance for end users isn’t necessarily better than TCP.
The performance of QUIC is a whole other story, however, it’s here to stay and it’s slowly starting to gain traction. There are many implementations for various web servers, none of them are getting it right at the moment, but the fact that QUIC support in nginx is still experimental in 2024 is a bit worrying to me.
why do you say that QUIC is here to stay and starting to gain traction if no web servers properly support it? why is it worrying for adoption to be slow if there’s no consensus that it’s an improvement?
QUIC is definitely here to stay because too many companies are investing more money into QUIC deployments and further R&D to make it better. One of the reasons why the adoption is slow is that it’s such a radical departure from the whole TCP/IP architecture and it turns out that network stacks and a bunch of other software was simply never that good in handling large UDP traffic so far, simply because there was no need to do so.
It’s going to take a while before everyone in the hierarchy gets things right, but based on what I see in the industry overall, we’re slowly starting to see some kind of resemblance of the light at the end of the tunnel. In fact, just today a very nice comment appeared on the site which sums some of the major challenges quite nicely.
That was very informative; thanks.
Honestly, “in 2024,” it sounds like all QUIC implementations are experimental and the worrying thing would be if any of them were not labeled as such.
I expect QUIC to become widely deployed but I was recently pondering having HTTP/3 on my servers and immediately stopped because I had no idea what to use. This makes me wonder if it’s ever going to replace HTTP/1 for most servers or if it will only be used by the largest players.
warp, via this extension: https://hackage.haskell.org/package/warp-quic
Nginx still doesn’t properly proxy provisional responses, which prevent the use of 103 early hints: https://forum.nginx.org/read.php?10,293049
Technically it’s not even a new feature, they were already defined in HTTP/1.0 RFC 1945 in 1996.
Huh, I didn’t realize HTTP/2 Push was so unpopular/underutilized. Intuitively, it makes sense – when you’re initially serving the index page for a SPA, you know the client is about to request your CSS and JS, so you might as well save a roundtrip. I guess it doesn’t make enough of a difference for most people to bother. But what do I know?
I guess it’s just really hard to implement. If you’re a web server, you would need to parse the HTML to know what resources would be fetched, and then what about resources fetched through JavaScript? That’d basically need the framework to push out the resources to the web server, but that wouldn’t work for e.g. static sites or CDNs (which are hosting ~all of the stuff you’d use HTTP/2 for). Turns out rel=“preload” combined with HTTP/3 works well enough without requiring web servers to implement an HTML parser.
Its latency wins are also a bit dubious because you may be forcing the retrieval of something the client already has in cache, and thus the various early hint mechanisms instead - the client knows its own cache contents.
In the end 103 Early Hint while theoretically sligthly less efficient is much simple, and also retro-compatible with HTTP/1.1 which is nice.
In addition to requiring coupling between the pages/application and the web server (which might be a reverse proxy and therefore separate!), it wasn’t bringing much IIRC.
The alternative is having your browser parse the HTML it receives, start parsing it (which it does very well), spot resources it will require, and request them. Since the resources are often listed at the beginning of the HTML, the browser will do that quickly and often there isn’t really a gap.
Moreover, I think it could actually lead to a lower performance (too many elements listed, some unused, …)
That’s not ideal for index size. Would be better to have a str <-> int translation.
hmm yeah maybe room for an integer id option. eg https://sqids.org/ruby or similar. i think in most cases i’d rather have a big index than two ids though. it’s nice being able to drop into psql and search by the same id that’s in a url.
I didn’t mean to have two ids, but to have a way to decode these string ids into 64 or even 128but integers so they are stored as such in the DB, like UUIDs basically
do you know if you could have it produce the non integer representation with a select * right in postgres somehow? or would that require passing it through a function (or maybe a virtual column or something like that)
Yeah it’s very likely possible. But since your article is about Rails it’s much more easily done with Active Model attributes API.