The main argument against is that even if you assume good intentions, it won’t be as close to production as an hosted CI (e.g. database version, OS type and version, etc).
Lots of developers develop on macOS and deploy on Linux, and there’s tons of subtle difference between the two systems, such as case sensitivity of the filesystem, as well as default ordering just to give an example.
To me the point of CI isn’t to ensure devs ran the test suite before merging. It’s to provide an environment that will catch as many things as possible that a local run wouldn’t be able to catch.
To me the point of CI isn’t to ensure devs ran the test suite before merging.
I’m basically repeating my other comment but I’m amped up about how much I dislike this idea, probably because it would tank my productivity, and this was too good as example to pass up: the point of CI isn’t (just) to ensure I ran the test suite before merging - although that’s part of it, because what if I forgot? The bigger point, though, is to run the test suite so that I don’t have to.
I have a very, very low threshold for what’s acceptably fast for a test suite. Probably 5-10 seconds or less. If it’s slower than that, I’m simply not going to run the entire thing locally, basically ever. I’m gonna run the tests I care about, and then I’m going to push my changes and let CI either trigger auto-merge, or tell me if there’s other tests I should have cared about (oops!). In the meantime, I’m fully context switched away not even thinking about that PR, because the work is being done for me.
You’re definitely correct here but I think there are plenty of applications where you can like… just trust the intersection between app and os/arch is gonna work.
But now that I think about it, this is such a GH-bound project and like… any such app small enough in scope or value for this to be worth using can just use the free Actions minutes. Doubt they’d go over.
any such app small enough in scope or value for this to be worth using can just use the free Actions minutes.
Yes, that’s the biggest thing that doesn’t make sense to me.
I get the argument that hosted runners are quite weak compared to many developer machines, but if your test suite is small enough to be ran on a single machine, it can probably run about as fast if you parallelize your CI just a tiny bit.
With a fully containerized dev environment yes, that pretty much abolish the divergence in software configuration.
But there are more concern than just that. Does your app relies on some caches? Dependencies?
Where they in a clean state?
I know it’s a bit of an extreme example, but I spend a lot of time using bundle open and editing my gems to debug stuff, it’s not rare I forget to gem pristine after an investigation.
This can lead me to have tests that pass on my machine, and will never work elsewhere. There are millions of scenarios like this one.
I was once rejected from a job (partly) because the Dockerfile I wrote for my code assignment didn’t build on the assessor’s Apple Silicon Mac. I had developed and tested on my x86-64 Linux device. Considering how much server software is built with the same pair of configurations just with the roles switched around, I’d say they aren’t diminished enough.
Was just about to point this out. I’ve seen a lot of bugs in aarch64 Linux software that don’t exist in x86-64 Linux software. You can run a container built for a non-native architecture through Docker’s compatibility layer, but it’s a pretty noticeable performance hit.
One of the things that I like having a CI is the fact that it forces you to declare your dev environment programmatically. It means that you avoid the famous “works in my machine” issue because if tests works in your machine but not in CI, something is missing.
There are of course ways to avoid this issue, maybe if they enforced that all dev tests also run in a controlled environment (either via Docker or maybe something like testcontainers), but it needs more discipline.
This is by far the biggest plus side to CI. Missing external dependencies have bitten me before, but without CI, they’d bite me during deploy, rather than as a failed CI run. I’ve also run into issues specifically with native dependencies on Node, where it’d fetch the correct native dependency on my local machine, but fail to fetch it on CI, which likely means it would’ve failed in prod.
This is something “local CI” can check for. I’ve wanted this, so I added it to my build server tool (that normally runs on a remote machine) called ding. I’ll run something like “ding build make build” where “ding build” is the ci command, and “make build” is what it runs. It clones the current git repo into a temporary directory, and runs the command “make build” in it, sandboxed with bubblewrap.
The point still stands that you can forget to run the local CI.
I’m not gonna lie, the three letter name is a compelling reason.
And I hope it’s coming across that I’m not ascribing any kinda intent to you. I think it makes sense as a name for the tool and I think that it might cause some who’s-on-first type shenanigans.
You’re using that term incorrectly, this is a better example of “de jure,” in that a person has by decree seized ownership of a generic term. De facto would be like, I dunno, WinRAR being the de facto archive utility for Windows.
Sort was released 53 years ago, we’ve learned lessons since then. This project takes ownership of a term used by millions (billions?) of people. At the time sort was written the author probably had the phone number of every possibly immediately impacted person.
At least with tools it’s your choice whether to introduce the tool into your PATH. But the phenomenon is a continual annoyance to me in package managers without mandatory namespacing. I don’t know if it’s hubris or ignorance that leads people to do this. But they do, so if you’re making a package repository, I beg of you, require namespacing.
Just yesterday I wanted a nice wrapper for bitfields in Rust and found the bitfield crate, which provides a somewhat inelegant macro, and doesn’t actually work with another crate that it claims to support. There is a bitfield-struct crate that is a bit better, possibly named that because the other one already took bitfield. But there just shouldn’t be a way for someone to squat on the name bitfield.
I’m not bothered because it doesn’t strike me as some moneymaking land-grab. I’d rather the name be used by this agnostic tool than a particular company’s product.
However I do have to disambiguate a little bit every time I talk about it, by calling it “simonw’s llm” or “the pypi package llm.” That’s cumbersome.
It seems silly for me to say this in a discussion about words but: I don’t think you quibbling over the word harm is productive, and I do think it’s appropriate. This doesn’t just make it confusing to discuss the tool, it makes it confusing to NOT discuss the tool. The overload is present when discussing the concept, as well.
The confusion isn’t inherent to the utility, it’s a choice. It’s optional. You only have to say more words to speak clearly about it because it is unclear for no good reason. Thing coulda been called llmer, llmo, llimo (gets you there in style), whatever. Just some differentiating mouth noises.
When I started reading the article I thought I was getting a tutorial, about the middle I realized I was getting an explanation of the intent behind a shared repository. The word misleading feels judgmental, like I’m ascribing negative intent, but I don’t think it’s that. Could use some editing, imo.
You misunderstood. The first project started in like 2005, the second in 2014. They both have over a decade of use and both continue to be used. Neither is gonna be renamed.
No. I didn’t misunderstand. Yes, both have continued to exist largely without problems, but the fact that we have two, independent, package managers named “asdf” is just stupid. In 2014, a quick google search for “asdf” would have easily found asdf for Common Lisp. Why’d we get here in the first place?
If it wasn’t another package manager, I’d be more inclined to not care. But, come on.
And yes, the ship has sailed for asdf — but my Lua will be a general - general scripting language! It’s not just a general embeddable scripting language… No. It takes scripting languages to a whole new level of general-ness!
edit to say: 2005 actually seemed too late. Turns out asdf is 24 years old.
It’s true that asdf for Common Lisp is not a package manager — it’s a build tool that… creates packages.
The other asdf swaps out “runtimes” … a so called “version manager” allowing you to ensure that the right set of tools are being used when you go to develop, build, etc. It can install runtimes, and it can remove runtimes. I mean, I don’t want to go against the wishes of the author… but if it looks like a duck…
But, I’ll choose to concede to your point that neither of these are de facto “package managers,” but it’s hard to argue against the fact that they are in the same general purpose space.
edit to say: 2005 actually seemed too late. Turns out asdf is 24 years old.
Yes, I remember circa 2008 or 2010 when Faré revamped it as asdf2. But what’s the point? So there’s a namespace collision across two systems. No Lisper is going to mistake it.
I will not claim to be a Lisper, but having used neither in anger I was for a time under the vague misapprehension that there was only one asdf and that the Lisp one could be (ab)used to install non-Lisp things, kind of like pip sometimes is.
I fully admit this was an entirely a me problem and if I ever tried to use either my confusion would have been resolved, but there is my sample of 1 on the confusion being possible.
The point is that if you’re starting a new project, maybe do a quick search to see if something else by the same name is already established? This isn’t hard. It’s common courtesy. I don’t see why it’s controversial?!
For kicks I checked Wikipedia: asdf has even had a Wikipedia page since 2007.
Sure, I agree and would do the same, but someone didn’t and they made free software and put it out there for other people to use and continued to maintain it. I can’t get my nerd rage on about the accidental reuse of a project name.
Wait until you find out that Java was named after a coffee which was named after an island. Or that Python was named after a comedy skit which was named after a snake. And don’t get me started on Go or Rust or Ruby! It’s a miracle that we can get anything done in the software world with all of this semantic ambiguity! May we discuss nothing else except this endlessly fascinating topic until the end of time! /s
Wait until you find out that Java was named after a coffee which was named after an island. Or that Python was named after a comedy skit which was named after a snake. And don’t get me started on Go or Rust or Ruby! It’s a miracle that we can get anything done in the software world with all of this semantic ambiguity! May we discuss nothing else except this endlessly fascinating topic until the end of time! /s
You really got me… I’ll give you another, the function car in Lisps always confuses me with the car that you drive on the road. Surely we must move to unique identifiers every time we need to name something! /s
Or, you know, you can just stop naming the two things in the same category of things the same name. That might work. The lack of common decency is crazy.
Or, you know, you can just stop naming the two things in the same category of things the same name. That might work. The lack of common decency is crazy.
Obviously taxonomies are arbitrary and subjective, and a runtime version manager is not obviously in the same category as lisp build tooling. You could argue that they are both in the category of “software” just like you could argue that Rust (programming language) and Rust (game), yet people on this forum don’t routinely drag that up in every post with the rust tag because there’s obviously no such moral prohibition against using pedestrian names across multiple software projects.
Looks like it would be totally locked to bicameral scripts. Kinda surprised to not see IDs like characters prefixed with a # sigil or something to allow more languages.
“bicameral scripts” are writing systems that have two cases, such as the English alphabet with its lower and UPPER cases, or Greek, or Cyrillic. CJK scripts, for example, do not have this distinction.
Second, this is one giant “init” commit, with no way to figure out how the code evolved, what was the reason for a change/line of code (thanks to git blame).
I’m often sad that for many people “FLOSS” means “uploaded to a public github repository” :(
They’re pretty upfront about the intentions in the very first paragraph of the readme:
The codebase is provided as a reference for those interested in learning how Campsite works. We welcome forks of this repository for use in non-commercial projects.
It was 100% close-source privately-funded codebase and they’re releasing it for other people to learn from and self-host. None of us were owed this, frankly, and I applaud them for doing it.
Isn’t it a possible privacy issue if you include usernames and commit messages? Couldn’t an EU based individual complain about their data being in there? They could have also been doing very embarrassing things like keeping secrets in code?
I’d rather a squashed commit than never seeing the source.
Yeah, open sourcing out of a corporate environment with the full commit history is an enormous amount of work for an unclear amount of value for the organization.
If you don’t like it, you should write from scratch your own team messaging platform, and share it under a permissive license. The authors have no moral, ethical, or legal obligation to let you make commercial derivatives of the software they’ve shared with the public.
If you torture yourself to achieve miracles at work, you can look forward to that level of performance being considered your new “normal”. You are rarely if ever rewarded in any way- with money or loyalty- for sacrificing your physical health and mental wellbeing, and for what- deadlines and resource constraints often made up from thin air at the whims of executives? Don’t let anyone convince you that crunch is a law of nature.
And if you torture yourself to achieve miracles, at least make sure that you own the fruit of your labor. If your salary is fixed, your effort should be too.
i haven’t really had that problem. but i also tend to sleep through (low stakes) meetings after a burst, so maybe that dials the expectations back down
my take is that you set the new normal high if you crunch, and also try to pretend it’s not hard. or aren’t willing to take time back afterward, etc.
(i think most people, in most employee-manager relationships, aren’t willing/able to do this effectively)
If you are a visible hero, you will likely get recognition.
This happened to me, and is why I am a non-founding CTO now. Someone recalled that I’m the kind of person who “works miracles” and put my name forward for consideration along with a recommendation.
The same miracle work gained me other jobs where I basically walked in, however, toiling away quietly or only complaining wasn’t the way it worked for me; and because it worked for me doesn’t mean it’ll work for you. Just an anecdote.
My person you really only said “nuh-uh plus I’m old,” there isn’t even an anecdote here. Like, I’m not even sure which facet of their comment you’re disagreeing with.
It’s incredible how any suggestion that they pay money out of their own pockets to help maintain a tool that is critical for them, just seems to roll off these ultra-rich finance firms, like water off a seal’s back. They keep making the same argument on auto-pilot while pretending not to notice the incongruency.
That doesn’t even ability in this case. Their salary doesn’t depend on the rounding error of contributing to Git, and they pay for lots of things. They’re just being cheap. (Or, to put it in finance terms, there’s no ROI to contributing, if you’re already getting it for free.)
I’m skeptical about adding Rust to established C projects like git and curl, when they have Rust-native alternatives.
Major rewrites will be met with resistance. It may take a long time before any substantial portion of git is written in Rust. Additionally, mixed C+Rust projects may not want to let go of C compatibility, and likely feel non-idiomatic from Rust’s perspective.
OTOH there’s the gix reimplementation project that is Rust-based from ground up. It doesn’t need to convince anybody. It won’t bother anyone’s niche platform. Rust doesn’t need the original git to be rewritten, it’s enough to push gix past the finish line.
The thing is, git isn’t really a C program, it is a collection of programs, written in various languages, that happen to interoperate though a common set of file formats.
As discussed in the mailing list thread, this makes it really easy to incrementally adopt a new language like Rust: you just make new commands in the new language. No re-writing neccesary.
It might be educational for the audience to list out these programs
git, like many command line tools, turns git foo into an invocation of git-foo. Furthermore, commands are split into two categories: “plumbing” commands which do low-level things, and “porcelain” commands which are intended for end-users.
My understanding is that in the old days, various commands tended to be distributed as literally individual programs, but more recently, they’ve rolled them up into a single executable. This has been called the “lib-ification”, as mentioned in the mailing list thread.
Regardless, the main three implementation languages have been C, perl, and shell. For example:
Thus, Git should probably follow the same approach of not converting the
already existing code, but frankly, I don’t see what would actually be
the “new leafs” written in Rust.
One response:
In addition to some of the examples Junio mentioned elsewhere, I think
new toplevel commands, like git-replay, would qualify.
Yes, but the problem is that once you introduce Rust into anything, you get Rust people, and once you have Rust people involved, your project’s objectives no longer have anything to do with whatever the project actually does.
I’m not sure this is an entirely fair way of thinking about it. There are a lot of Rust developers who see Rust as a tool to solve problems instead of a prescription worldview.
Not to mention, there’s not that much need to be worried about interoperability if it’s file format based.
Rust doesn’t need the original git to be rewritten, it’s enough to push gix past the finish line.
I’m not sure this is true. If gix is trying to match git, it will always be behind on compatibility (both features and bugs), and will always be a second choice. Playing catch-up to support a format controlled by someone else is not a path to broad success; just look at LibreOffice.
It will have to keep up with the git protocol and the git disk format, but this seems manageable? libgit2 has reimplement enough of git to power a lot of git-related tools (maybe even the majority).
OTOH if the Rust version works out, and will be easier to extend, then maybe the original git will have to catch up with features? :)
As an exercised in understanding this a bit better, I took a look rewriting the original git commit from Linus in Rust the other day. The C idioms used for managing files and for global shared state are pretty crappy in comparison from what you get out of the box from Rust. Abstracting away from dealing with file descriptors, scanf, pointer arithmetic, etc. is a good thing IMO. Then you get library support by importing things like sha1_smol instead of writing your own things. Even looking at the most recent git code shows that the C code is often pretty low level.
I actually tried to do this a few years ago! I got burned out trying to untangle all the global mutable state. And I found a few segfaults too. Good for you!
Lemme immediately undermine my argument by acknowledging zero day exploits as a concept, but…
I genuinely cannot imagine that, once gitoxide is in keep-up mode, that I will adopt a new git feature before they implement it, unless they give up. And if that happens, or if that delay does exist, I can just use the real thing for a bit. I mean, if gix offered something over git, maybe it’s faster or something for certain operations, someone could wrap them both and choose which to call for a given command.
But that wouldn’t influence the development of git unless they actually diverged, in which case they’re not improving git and they’re fighting for mindshare.
What is the broad impact of dropping support for a platform that has a significant investment in git, where loss of support could have societal impact. There are platforms with 100 million to 1000 million lines of code managed by git today. This type of investment-related impact is specific to git compared to other Open-Source products. Leaving debit or credit card authorizers without a supported git would be, let’s say, “bad”.
I feel like people value git radically more than they value these platforms. The result wouldn’t be “those platforms drop git” it would be “everyone drops those platforms”.
Nah. What? Real time payment processor core infrastructure isn’t gonna swap operating systems so they can keep their VCS. That’s genuinely silly. What “everyone” will do is not notice or care, because there’s a small number of people who would actually be impacted by this, and I trust them to figure it out.
SQLite is like three guys. I am confident in those three guys ability to deliver safe and well tested C code. These Turso folks think SQLite would benefit from external contributions. I am less confident in the ability of new external contributors to do that.
SQLite doesn’t have a safety problem, libSQL does. So if you’re walking around going “gosh, I sure would love to switch from SQLite to libSQL, but I’m just not confident in its memory safety” then Limbo might be for you.
SQLite doesn’t have a safety problem, libSQL does. So if you’re walking around going “gosh, I sure would love to switch from SQLite to libSQL, but I’m just not confident in its memory safety” then Limbo might be for you.
That’s a very unfair take of the article. Article nowhere mentions that SQLit has a safety problem.
We are rewriting because we want to have AsyncIO, WASM first and support Deterministic Simulation Testing support from the beginning.
It was more of a logical sequence than an accusation. C code is often unsafe. SQLite devs have demonstrated they can produce safe code, so that isn’t an issue for it. libSQL intends to have many more contributors, so I wouldn’t feel comfortable making the same assumption.
While I’m not enthused at having another thing wanting to pull an async runtime into my sync projects, once I feel there’s been enough time for Limbo to have proven itself as trustworthy as SQLite, I look forward to using it.
SQLite’s dependence on a C toolchain for more than just libc makes statically linked musl-libc builds annoyingly inconvenient compared to a pure Rust codebase.
…maybe, by the time that happens, Rust will have something like keyword generics and Limbo will have adopted them enough that it can be used synchronously without hiding an async runtime behind the scenes.
I think you’re being unduly negative. Async IO does not require an async runtime (like tokio) or even async/await. I quickly searched for async in the limbo github repo and it does not look like they are using either.
Limbo is designed to be fully asynchronous. SQLite itself has a synchronous interface, meaning driver authors who want asynchronous behavior need to have the extra complication of using helper threads.
It doesn’t make sense for something to be async internally if downstreams depending on it still have to block on getting a result, so the absence of async fn suggests they either haven’t implemented their higher-level API for Rust consumers yet or they plan on some kind of ugly, NIHed alternative for Rust downstreams that do want to access it asynchronously.
Fair point… but not what I meant to be getting at in this specific case.
SQLite could already do that. When they say “SQLite itself has a synchronous interface, meaning driver authors who want asynchronous behavior need to have the extra complication of using helper threads.”, they’re talking about what external API SQLite exposes.
I’m not sure what you’re saying. You seem to be complaining that there’s an async interface in Limbo but you don’t want that, and I showed that you don’t need to - you can just block. Now you seem to be saying something else but I’m not sure what.
I’m saying that their stated intent makes no sense if they don’t either expose or plan to expose an async interface to consumers.
My assumption was that they already do, and given that it makes no sense to expose an async interface to downstream Rust without defining the API in terms of async fns, if you’re going to expose async fns, you might as well use async/await in your implementation. I admit I was wrong there.
Now I’m just hoping that, if/when they do offer and async interface, it’ll remain optional.
It’s an interesting claim. External contributors may not have the obsession with stable database formats that a small group dedicated to the cause might.
The right way to think about it is that SQLite would benefit from competition. Personally I believe that this competition would be better to come from a re-implementation than a fork. So I think this is good. Will it succeed? Will I like it? Will I use it? I mean probably the answer is: no, but I sure as hell will applaud that effort.
By watching Not Just Bikes youtube videos I learned about induced demand in transportation: increasing road capacity does not result in reduced congestions. In response to the added capacity, more people decide to drive a car, and sit in the same and even larger congestions.
Wouldn’t something similar apply here? Making programs run faster may not result in less CPU time spent. An optimized game engine will load cut scenes faster, and make the game more addictive. A faster JS engine will allow adtech companies to send even more junk with every page load. A leaner linux kernel will allow it to run on smaller chips, and we will get IoT crap on more appliances. And so on.
I’m not saying we shouldn’t try to make things more efficient. I’m saying, this could potentially result in more computation (and energy spent), not less.
Yes, the same thing will happen. In the context of resource efficiency, the phenomenon is called the Jevons Paradox. Making a resource’s usage more efficient at a societal level causes more of it to be used, not less. Faster software won’t reduce emissions, it will just make us run more software.
Induced demand only applies when people actually want something. People want to drive, so when driving becomes cheaper, more people will drive. Only because of the large quantity of people who want to drive somewhere combined with the geometry of the world, this results in congestion never being fixed.
For an example of induced demand being good, look at trains. If you make trains run more often, they may become more popular, as they are now more useful for getting between places. They may become crowded, but they’ll get to their destinations just as fast. And if the trains are getting quite crowded, well, that’s a good argument for running more trains, increasing frequency, which makes the service better yet again, and may induce even more demand. So, the service becomes better with induced demand in one axis (frequency) but not necessarily another (crowding). But there’s another crucial difference: the space savings of trains compared to cars is so great, that it actually is possible to fully meet all demand, so inducing demand until trains are too crowded isn’t inevitable, the way congestion is inevitable with cars.
Finally, we get to efficient programs and computers. More efficient programs probably would induce demand. But how much? It can only happen to the extent that there is demand for running programs currently not met because it’d be too expensive. How large is that extent? It is surely not infinite. How much more efficient can we make our programs? The question of whether we’d use more energy or less depends on the answers to those questions.
People want to drive, so when driving becomes cheaper, more people will drive.
I mean, they want to get to their destination, they believe driving is the only valid way to do that, and they reason that more lanes means more ez ride.
It’s frustrating because (here in Chicago) it is absolutely true that trains are late, busses bunch up on routes, and scheduled routes are sometimes too infrequent, but folks who use them as reason not to use transit ignore the indignities they already experience while driving. Oh well.
Am I off topic? At the very least I’m looking too far into the analogy. Pardon me.
Why tho? When do I need a number without any further context? SEVEN is exactly as magic as 7, and DAYS_IN_WEEK = SEVEN doesn’t offer anything over assigning directly to the integer while adding the expense of understanding this here thing.
Unless we’re just having fun, in which case I’m into it, but I think it’d be nice to indicate it’s a parody library. This does seem to be a good way to achieve dogmatic adherence to the “best practice” of not assigning ints and could fool junior folks into thinking it’s the right move. Teach through humor rather than making folks feel foolish.
Yep. 42 is meaningless, ANSWER_TO_THE_GALAXY is not. FOURTY_TWO contains identical information to the former, except… actually not identical: it’s slower to parse when dealing with these small numbers, especially if English isn’t your first/native language. Maybe on the way high end of the number scale this starts making more sense (ONE_BILLION is actually somewhat useful maybe?)
Steamtinkerlaunch is a wrapper for a bunch of tools supporting Steam games on Linux. It has a bunch of support files but the main script is ~27k lines of bash. I tried digging around in it to fix some issues I was having and the experience made me genuinely unhappy. Ruined my day.
Inverse Sequencing: Store a counter starting from the maximum value (u64::max) and decrement it with each insert. The S3 LIST API is lexicographical, so you always get the last inserted filename with a single list call. I am split on this solution as I find it cognitively taxing.
As soon as you pointed it out, this seemed the obvious best solution. Sort of a workaround for LIST’s inability to reverse the sort order.
Also, if you don’t want the application programmer to have to know that things are stored in reverse order then you can just have some wrapper function that subtracts the user-visible sequence number from u64::max.
The initial juxtaposition between the two Hello World’s is not v good. The one that imports the color library outputs colored text, while the one that does not, does not. It shows why you would import something, to get behavior not offered, or not offered in a straightforward manner, by the stdlib. A better example would be http.serveMux vs gorilla/mux.
While it does directly reference Android I omitted the tag, as I believe the meat of this post is adapting a lipo device to direct USB for safe upcycling.
So merry ChristMatz 2022, we got 3.2 where YJIT is stable but off by default. 2023 it 3.3, YJIT’s on by default in a paused state so you can enable it in code, rather than when compiling and passing a flag to executables. No relevant change this year.
Anyone got a sense of when it’ll be on by default? Or remove the flag altogether? Why might someone avoid YJIT, and why can’t we give em an LTS version?
I dunno, seeing all that feature flagging in the low level implementation puts a squirm in my spine. I don’t think it’s like wrong, I just wouldn’t want to be maintaining it. OTOH, that’s all focused and prolly low churn stuff, so I suppose it ain’t hurting no one.
I believe another reason it is still off by default is more people-related: ruby-core is mostly Japanese and C-based. YJIT is maintained by Shopify employees and Rust-based.
Ruby has many of the same problems as Linux – some groups want to move to Rust for safety and speed but the existing maintainers often don’t know Rust at all.
Ruby has improved quite a lot. I love Ruby and used it extensively in the late 2000s. MRI was a disaster. Slow and had memory leak issues in long-running processes. Right now, it’s much faster and more stable. Night and day difference. Leaving Ruby aside, it never ceases to amaze me how good and performant the JVM is. The results in this benchmark are quite stunning.
yea, it’s a great question, and something I thought of mentioning in the article (then forgot).
I think the primary reason it’s still off by default is because of memory overhead. any JIT will usually add a not insignificant amount of memory usage on top of the runtime.
That said, YJIT is really configurable about the amount of memory it will consume. By default I think it’s 48 megs per process max? I know Github uses it but tunes it down a bit to more like 16megs. So possibly in the future it will be on, but by default set to a lower max.
Would be curious to hear from the YJIT team on their thoughts on that!
48MiB is how much executable memory it will use, but it also has to keep track of some metadata which usaly account for 2 or 3x the executable memory. So you’ll easily end up with a 150-200MB overhead with the default settings.
And yes you are right, the main blocker for enabling it by default really is extra memory usage, even though the YJIT team never formally proposed it, but from informal discussion with other Ruby committers, it’s clear that it would be the blocker.
I think the main reason to not flip it on by default in 3.3+ was that it could break container / memory-constrained environments if folks upgrade blindly and don’t monitor and increase mem limits appropriately It also can increase startup time for interactive use cases, such as CLIs and such.
I dunno if that was really the right call, but it seems that more conservative approach still holds: I haven’t heard any change in the default YJIT settings for 3.3+.
I think this is a great idea, but I am anticipating folks explainIng why it isn’t.
The main argument against is that even if you assume good intentions, it won’t be as close to production as an hosted CI (e.g. database version, OS type and version, etc).
Lots of developers develop on macOS and deploy on Linux, and there’s tons of subtle difference between the two systems, such as case sensitivity of the filesystem, as well as default ordering just to give an example.
To me the point of CI isn’t to ensure devs ran the test suite before merging. It’s to provide an environment that will catch as many things as possible that a local run wouldn’t be able to catch.
I’m basically repeating my other comment but I’m amped up about how much I dislike this idea, probably because it would tank my productivity, and this was too good as example to pass up: the point of CI isn’t (just) to ensure I ran the test suite before merging - although that’s part of it, because what if I forgot? The bigger point, though, is to run the test suite so that I don’t have to.
I have a very, very low threshold for what’s acceptably fast for a test suite. Probably 5-10 seconds or less. If it’s slower than that, I’m simply not going to run the entire thing locally, basically ever. I’m gonna run the tests I care about, and then I’m going to push my changes and let CI either trigger auto-merge, or tell me if there’s other tests I should have cared about (oops!). In the meantime, I’m fully context switched away not even thinking about that PR, because the work is being done for me.
You’re definitely correct here but I think there are plenty of applications where you can like… just trust the intersection between app and os/arch is gonna work.
But now that I think about it, this is such a GH-bound project and like… any such app small enough in scope or value for this to be worth using can just use the free Actions minutes. Doubt they’d go over.
Yes, that’s the biggest thing that doesn’t make sense to me.
I get the argument that hosted runners are quite weak compared to many developer machines, but if your test suite is small enough to be ran on a single machine, it can probably run about as fast if you parallelize your CI just a tiny bit.
I wonder if those differences are diminished if everything runs on Docker
With a fully containerized dev environment yes, that pretty much abolish the divergence in software configuration.
But there are more concern than just that. Does your app relies on some caches? Dependencies?
Where they in a clean state?
I know it’s a bit of an extreme example, but I spend a lot of time using
bundle openand editing my gems to debug stuff, it’s not rare I forget togem pristineafter an investigation.This can lead me to have tests that pass on my machine, and will never work elsewhere. There are millions of scenarios like this one.
I was once rejected from a job (partly) because the Dockerfile I wrote for my code assignment didn’t build on the assessor’s Apple Silicon Mac. I had developed and tested on my x86-64 Linux device. Considering how much server software is built with the same pair of configurations just with the roles switched around, I’d say they aren’t diminished enough.
Was just about to point this out. I’ve seen a lot of bugs in aarch64 Linux software that don’t exist in x86-64 Linux software. You can run a container built for a non-native architecture through Docker’s compatibility layer, but it’s a pretty noticeable performance hit.
One of the things that I like having a CI is the fact that it forces you to declare your dev environment programmatically. It means that you avoid the famous “works in my machine” issue because if tests works in your machine but not in CI, something is missing.
There are of course ways to avoid this issue, maybe if they enforced that all dev tests also run in a controlled environment (either via Docker or maybe something like testcontainers), but it needs more discipline.
This is by far the biggest plus side to CI. Missing external dependencies have bitten me before, but without CI, they’d bite me during deploy, rather than as a failed CI run. I’ve also run into issues specifically with native dependencies on Node, where it’d fetch the correct native dependency on my local machine, but fail to fetch it on CI, which likely means it would’ve failed in prod.
Here’s one: if you forget to check in a file, this won’t catch it.
It checks if the repo is not dirty, so it shouldn’t.
This is something “local CI” can check for. I’ve wanted this, so I added it to my build server tool (that normally runs on a remote machine) called ding. I’ll run something like “ding build make build” where “ding build” is the ci command, and “make build” is what it runs. It clones the current git repo into a temporary directory, and runs the command “make build” in it, sandboxed with bubblewrap.
The point still stands that you can forget to run the local CI.
What’s to stop me from lying and making the gh api calls manually?
I am bothered by this llm tool taking its name directly from the generic concept.
When I released the first version in in April 2023 the acronym LLM was still pretty obscure.
I was excited to find a three letter tool name that was still available on PyPI.
I’m not gonna lie, the three letter name is a compelling reason.
And I hope it’s coming across that I’m not ascribing any kinda intent to you. I think it makes sense as a name for the tool and I think that it might cause some who’s-on-first type shenanigans.
I remember when it meant master of laws https://en.m.wikipedia.org/wiki/Master_of_Laws
This stops making sense if you take into consideration most classic UNIX utilities.
Why can’t we have a de-facto utility for interacting with LLMs?
You’re using that term incorrectly, this is a better example of “de jure,” in that a person has by decree seized ownership of a generic term. De facto would be like, I dunno, WinRAR being the de facto archive utility for Windows.
Sort was released 53 years ago, we’ve learned lessons since then. This project takes ownership of a term used by millions (billions?) of people. At the time sort was written the author probably had the phone number of every possibly immediately impacted person.
At least with tools it’s your choice whether to introduce the tool into your PATH. But the phenomenon is a continual annoyance to me in package managers without mandatory namespacing. I don’t know if it’s hubris or ignorance that leads people to do this. But they do, so if you’re making a package repository, I beg of you, require namespacing.
Just yesterday I wanted a nice wrapper for bitfields in Rust and found the
bitfieldcrate, which provides a somewhat inelegant macro, and doesn’t actually work with another crate that it claims to support. There is abitfield-structcrate that is a bit better, possibly named that because the other one already tookbitfield. But there just shouldn’t be a way for someone to squat on the namebitfield.I’m not bothered because it doesn’t strike me as some moneymaking land-grab. I’d rather the name be used by this agnostic tool than a particular company’s product.
However I do have to disambiguate a little bit every time I talk about it, by calling it “simonw’s llm” or “the pypi package llm.” That’s cumbersome.
That confusion is my concern. I’m not calling anyone an asshole, I think this harm is unintentional but it is still present.
I’m not sure I could call it harm on the basis of needing to say another word or two to speak clearly about something I find so helpful.
It seems silly for me to say this in a discussion about words but: I don’t think you quibbling over the word harm is productive, and I do think it’s appropriate. This doesn’t just make it confusing to discuss the tool, it makes it confusing to NOT discuss the tool. The overload is present when discussing the concept, as well.
The confusion isn’t inherent to the utility, it’s a choice. It’s optional. You only have to say more words to speak clearly about it because it is unclear for no good reason. Thing coulda been called llmer, llmo, llimo (gets you there in style), whatever. Just some differentiating mouth noises.
When I started reading the article I thought I was getting a tutorial, about the middle I realized I was getting an explanation of the intent behind a shared repository. The word misleading feels judgmental, like I’m ascribing negative intent, but I don’t think it’s that. Could use some editing, imo.
Cool demo, tho.
Asdf seems a bit overloaded: https://asdf.common-lisp.dev/ vs https://asdf-vm.com/
My guy the more recent of the two is 11ish years old, you are simply too late.
I shall declare the name of my new scripting language, Lua! The last version of the other Lua was months ago! Surely there will be no confusion!
eyeroll
You misunderstood. The first project started in like 2005, the second in 2014. They both have over a decade of use and both continue to be used. Neither is gonna be renamed.
Hope that helped.
No. I didn’t misunderstand. Yes, both have continued to exist largely without problems, but the fact that we have two, independent, package managers named “asdf” is just stupid. In 2014, a quick google search for “asdf” would have easily found asdf for Common Lisp. Why’d we get here in the first place?
If it wasn’t another package manager, I’d be more inclined to not care. But, come on.
And yes, the ship has sailed for asdf — but my Lua will be a general - general scripting language! It’s not just a general embeddable scripting language… No. It takes scripting languages to a whole new level of general-ness!
edit to say: 2005 actually seemed too late. Turns out asdf is 24 years old.
If you didn’t misunderstand my comment then you misrepresented it. That’s worse. Please stop trying to score points on people.
Neither project advertises itself as a package manager.
It’s true that asdf for Common Lisp is not a package manager — it’s a build tool that… creates packages.
The other asdf swaps out “runtimes” … a so called “version manager” allowing you to ensure that the right set of tools are being used when you go to develop, build, etc. It can install runtimes, and it can remove runtimes. I mean, I don’t want to go against the wishes of the author… but if it looks like a duck…
But, I’ll choose to concede to your point that neither of these are de facto “package managers,” but it’s hard to argue against the fact that they are in the same general purpose space.
Yes, I remember circa 2008 or 2010 when Faré revamped it as asdf2. But what’s the point? So there’s a namespace collision across two systems. No Lisper is going to mistake it.
I will not claim to be a Lisper, but having used neither in anger I was for a time under the vague misapprehension that there was only one asdf and that the Lisp one could be (ab)used to install non-Lisp things, kind of like pip sometimes is.
I fully admit this was an entirely a me problem and if I ever tried to use either my confusion would have been resolved, but there is my sample of 1 on the confusion being possible.
Oh, good, it wasn’t just me!
The point is that if you’re starting a new project, maybe do a quick search to see if something else by the same name is already established? This isn’t hard. It’s common courtesy. I don’t see why it’s controversial?!
For kicks I checked Wikipedia: asdf has even had a Wikipedia page since 2007.
Sure, I agree and would do the same, but someone didn’t and they made free software and put it out there for other people to use and continued to maintain it. I can’t get my nerd rage on about the accidental reuse of a project name.
Wait until you find out that Java was named after a coffee which was named after an island. Or that Python was named after a comedy skit which was named after a snake. And don’t get me started on Go or Rust or Ruby! It’s a miracle that we can get anything done in the software world with all of this semantic ambiguity! May we discuss nothing else except this endlessly fascinating topic until the end of time! /s
You really got me… I’ll give you another, the function
carin Lisps always confuses me with thecarthat you drive on the road. Surely we must move to unique identifiers every time we need to name something! /sOr, you know, you can just stop naming the two things in the same category of things the same name. That might work. The lack of common decency is crazy.
Obviously taxonomies are arbitrary and subjective, and a runtime version manager is not obviously in the same category as lisp build tooling. You could argue that they are both in the category of “software” just like you could argue that Rust (programming language) and Rust (game), yet people on this forum don’t routinely drag that up in every post with the
rusttag because there’s obviously no such moral prohibition against using pedestrian names across multiple software projects.The melodrama and bike-shedding are crazy.
Yeah, I was confused for a bit. This asdf-vm, from my reading, it’s like SDKMAN combined with <your lang’s hacked up env tool here> built in.
Looks like it would be totally locked to bicameral scripts. Kinda surprised to not see IDs like characters prefixed with a
#sigil or something to allow more languages.What?
“bicameral scripts” are writing systems that have two cases, such as the English alphabet with its lower and UPPER cases, or Greek, or Cyrillic. CJK scripts, for example, do not have this distinction.
I love this reply.
A post so good that I wish I needed it.
This is one of the worst way to “open source” something :/ .
First of all, this is under CC-SA-NC, which is not approved by the Open Source Initiative, and also not approved by the Debian Free Software Guidelines. CC-SA would be fine, even though it’s not great for code, but CC-NA-SA definitely breaks the freedom 2 & 3.
Second, this is one giant “init” commit, with no way to figure out how the code evolved, what was the reason for a change/line of code (thanks to git blame).
I’m often sad that for many people “FLOSS” means “uploaded to a public github repository” :(
They’re pretty upfront about the intentions in the very first paragraph of the readme:
It was 100% close-source privately-funded codebase and they’re releasing it for other people to learn from and self-host. None of us were owed this, frankly, and I applaud them for doing it.
Isn’t it a possible privacy issue if you include usernames and commit messages? Couldn’t an EU based individual complain about their data being in there? They could have also been doing very embarrassing things like keeping secrets in code?
I’d rather a squashed commit than never seeing the source.
Yeah, open sourcing out of a corporate environment with the full commit history is an enormous amount of work for an unclear amount of value for the organization.
Speaking of floss, my man here is a goddamn horse dentist. They didn’t have to release none of this. This is a good thing.
If you don’t like it, you should write from scratch your own team messaging platform, and share it under a permissive license. The authors have no moral, ethical, or legal obligation to let you make commercial derivatives of the software they’ve shared with the public.
If you torture yourself to achieve miracles at work, you can look forward to that level of performance being considered your new “normal”. You are rarely if ever rewarded in any way- with money or loyalty- for sacrificing your physical health and mental wellbeing, and for what- deadlines and resource constraints often made up from thin air at the whims of executives? Don’t let anyone convince you that crunch is a law of nature.
And if you torture yourself to achieve miracles, at least make sure that you own the fruit of your labor. If your salary is fixed, your effort should be too.
Absolutely agree. Also, big fan of wigglypaint.
i haven’t really had that problem. but i also tend to sleep through (low stakes) meetings after a burst, so maybe that dials the expectations back down
my take is that you set the new normal high if you crunch, and also try to pretend it’s not hard. or aren’t willing to take time back afterward, etc.
(i think most people, in most employee-manager relationships, aren’t willing/able to do this effectively)
If you are a visible hero, you will likely get recognition.
This happened to me, and is why I am a non-founding CTO now. Someone recalled that I’m the kind of person who “works miracles” and put my name forward for consideration along with a recommendation.
The same miracle work gained me other jobs where I basically walked in, however, toiling away quietly or only complaining wasn’t the way it worked for me; and because it worked for me doesn’t mean it’ll work for you. Just an anecdote.
My experience* says otherwise. Just my anecdote.
*: 30+ years in tech, 20 of which were in gamedev. Full blown graybeard at this point. /sigh
My person you really only said “nuh-uh plus I’m old,” there isn’t even an anecdote here. Like, I’m not even sure which facet of their comment you’re disagreeing with.
I disagree that extra, hard work won’t be rewarded. My long, personal experience (ie the anecdote) supports this.
Was it worth it? Would you do it again if given the opportunity to repeat?
Without doubt, yes. 100%. It worked out very well for me.
But FWIW, I try to live by a policy of no-regrets.
Without getting too personal: can you tell me a bit about how it worked out for you? I’d like to hear more about your experience.
Sure, but privately. DM me and let’s go form there. Happy to hop on a video call with you after Christmas or early in the New Year.
My experience* says otherwise. Just my anecdote.
*: 30+ years in tech, 20 of which were in gamedev. Full blown graybeard at this point. /sigh
It’s incredible how any suggestion that they pay money out of their own pockets to help maintain a tool that is critical for them, just seems to roll off these ultra-rich finance firms, like water off a seal’s back. They keep making the same argument on auto-pilot while pretending not to notice the incongruency.
That doesn’t even ability in this case. Their salary doesn’t depend on the rounding error of contributing to Git, and they pay for lots of things. They’re just being cheap. (Or, to put it in finance terms, there’s no ROI to contributing, if you’re already getting it for free.)
I’m skeptical about adding Rust to established C projects like
gitandcurl, when they have Rust-native alternatives.Major rewrites will be met with resistance. It may take a long time before any substantial portion of git is written in Rust. Additionally, mixed C+Rust projects may not want to let go of C compatibility, and likely feel non-idiomatic from Rust’s perspective.
OTOH there’s the
gixreimplementation project that is Rust-based from ground up. It doesn’t need to convince anybody. It won’t bother anyone’s niche platform. Rust doesn’t need the original git to be rewritten, it’s enough to pushgixpast the finish line.The thing is,
gitisn’t really a C program, it is a collection of programs, written in various languages, that happen to interoperate though a common set of file formats.As discussed in the mailing list thread, this makes it really easy to incrementally adopt a new language like Rust: you just make new commands in the new language. No re-writing neccesary.
It might be educational for the audience to list out these programs and what language they’re implemented in.
git, like many command line tools, turns
git foointo an invocation ofgit-foo. Furthermore, commands are split into two categories: “plumbing” commands which do low-level things, and “porcelain” commands which are intended for end-users.My understanding is that in the old days, various commands tended to be distributed as literally individual programs, but more recently, they’ve rolled them up into a single executable. This has been called the “lib-ification”, as mentioned in the mailing list thread.
Regardless, the main three implementation languages have been C, perl, and shell. For example:
git commit: https://github.com/git/git/blob/master/commit.cgit mergetool: https://github.com/git/git/blob/master/git-mergetool.shgit send-email: https://github.com/git/git/blob/master/git-send-email.perlThe discussion from the list about this idea, which you can see all on one page here: https://lore.kernel.org/git/ZZ77NQkSuiRxRDwt@nand.local/t/#u
One response:
Yes, but the problem is that once you introduce Rust into anything, you get Rust people, and once you have Rust people involved, your project’s objectives no longer have anything to do with whatever the project actually does.
Why the blatant bad faith?
Sounds awful. Where are there examples of this?
I’m not sure this is an entirely fair way of thinking about it. There are a lot of Rust developers who see Rust as a tool to solve problems instead of a prescription worldview.
Not to mention, there’s not that much need to be worried about interoperability if it’s file format based.
I’m not sure this is true. If
gixis trying to matchgit, it will always be behind on compatibility (both features and bugs), and will always be a second choice. Playing catch-up to support a format controlled by someone else is not a path to broad success; just look at LibreOffice.It will have to keep up with the git protocol and the git disk format, but this seems manageable? libgit2 has reimplement enough of
gitto power a lot of git-related tools (maybe even the majority).OTOH if the Rust version works out, and will be easier to extend, then maybe the original
gitwill have to catch up with features? :)That’s arguably already happening, with Jujutsu.
As an exercised in understanding this a bit better, I took a look rewriting the original git commit from Linus in Rust the other day. The C idioms used for managing files and for global shared state are pretty crappy in comparison from what you get out of the box from Rust. Abstracting away from dealing with file descriptors, scanf, pointer arithmetic, etc. is a good thing IMO. Then you get library support by importing things like sha1_smol instead of writing your own things. Even looking at the most recent git code shows that the C code is often pretty low level.
I actually tried to do this a few years ago! I got burned out trying to untangle all the global mutable state. And I found a few segfaults too. Good for you!
I’m at the untangling phase of this also :D
sha1_smol is not suitable for git because git does not use classic SHA-1, it uses a variant of SHA-1 with counter-cryptanalysis collision detection (see also, see also)
Yeah - the context I’m looking at is just the way the code parses hashes into a value. E.g. https://github.com/git/git/blob/e83c5163316f89bfbde7d9ab23ca2e25604af290/read-cache.c#L13-L80 Just using Digest::from_str(…) works much nicer.
But even if you rewrite it, the rust code is just better.
Lemme immediately undermine my argument by acknowledging zero day exploits as a concept, but…
I genuinely cannot imagine that, once gitoxide is in keep-up mode, that I will adopt a new git feature before they implement it, unless they give up. And if that happens, or if that delay does exist, I can just use the real thing for a bit. I mean, if gix offered something over git, maybe it’s faster or something for certain operations, someone could wrap them both and choose which to call for a given command.
But that wouldn’t influence the development of git unless they actually diverged, in which case they’re not improving git and they’re fighting for mindshare.
I feel like people value git radically more than they value these platforms. The result wouldn’t be “those platforms drop git” it would be “everyone drops those platforms”.
Nah. What? Real time payment processor core infrastructure isn’t gonna swap operating systems so they can keep their VCS. That’s genuinely silly. What “everyone” will do is not notice or care, because there’s a small number of people who would actually be impacted by this, and I trust them to figure it out.
Hm, that’s probably true. I suppose the way I see it is that either:
None of those situations seem like something it should be overly concerned with
What in the absolute hell?
I think my estimation of the SQLite team just shot down by like three notches. Reminds me of Oracle behavior.
There are multiple test suites both free and proprietary https://www.sqlite.org/testing.html
You either got some high resolution notches or some jerky knees.
SQLite is like three guys. I am confident in those three guys ability to deliver safe and well tested C code. These Turso folks think SQLite would benefit from external contributions. I am less confident in the ability of new external contributors to do that.
SQLite doesn’t have a safety problem, libSQL does. So if you’re walking around going “gosh, I sure would love to switch from SQLite to libSQL, but I’m just not confident in its memory safety” then Limbo might be for you.
I guess. I dunno. I’m just some guy.
That’s a very unfair take of the article. Article nowhere mentions that SQLit has a safety problem.
We are rewriting because we want to have AsyncIO, WASM first and support Deterministic Simulation Testing support from the beginning.
It was more of a logical sequence than an accusation. C code is often unsafe. SQLite devs have demonstrated they can produce safe code, so that isn’t an issue for it. libSQL intends to have many more contributors, so I wouldn’t feel comfortable making the same assumption.
While I’m not enthused at having another thing wanting to pull an async runtime into my sync projects, once I feel there’s been enough time for Limbo to have proven itself as trustworthy as SQLite, I look forward to using it.
SQLite’s dependence on a C toolchain for more than just libc makes statically linked musl-libc builds annoyingly inconvenient compared to a pure Rust codebase.
…maybe, by the time that happens, Rust will have something like keyword generics and Limbo will have adopted them enough that it can be used synchronously without hiding an async runtime behind the scenes.
I think you’re being unduly negative. Async IO does not require an async runtime (like tokio) or even async/await. I quickly searched for
asyncin the limbo github repo and it does not look like they are using either.I was working from lines like this.
It doesn’t make sense for something to be async internally if downstreams depending on it still have to block on getting a result, so the absence of
async fnsuggests they either haven’t implemented their higher-level API for Rust consumers yet or they plan on some kind of ugly, NIHed alternative for Rust downstreams that do want to access it asynchronously.Sure it does. A library internally could do something like:
And then the caller could just block on the two futures synchronously while internally it’s polled concurrently.
Fair point… but not what I meant to be getting at in this specific case.
SQLite could already do that. When they say “SQLite itself has a synchronous interface, meaning driver authors who want asynchronous behavior need to have the extra complication of using helper threads.”, they’re talking about what external API SQLite exposes.
I’m not sure what you’re saying. You seem to be complaining that there’s an async interface in Limbo but you don’t want that, and I showed that you don’t need to - you can just block. Now you seem to be saying something else but I’m not sure what.
I’m saying that their stated intent makes no sense if they don’t either expose or plan to expose an async interface to consumers.
My assumption was that they already do, and given that it makes no sense to expose an async interface to downstream Rust without defining the API in terms of
async fns, if you’re going to exposeasync fns, you might as well useasync/awaitin your implementation. I admit I was wrong there.Now I’m just hoping that, if/when they do offer and
asyncinterface, it’ll remain optional.It’s an interesting claim. External contributors may not have the obsession with stable database formats that a small group dedicated to the cause might.
The right way to think about it is that SQLite would benefit from competition. Personally I believe that this competition would be better to come from a re-implementation than a fork. So I think this is good. Will it succeed? Will I like it? Will I use it? I mean probably the answer is: no, but I sure as hell will applaud that effort.
By watching Not Just Bikes youtube videos I learned about induced demand in transportation: increasing road capacity does not result in reduced congestions. In response to the added capacity, more people decide to drive a car, and sit in the same and even larger congestions.
Wouldn’t something similar apply here? Making programs run faster may not result in less CPU time spent. An optimized game engine will load cut scenes faster, and make the game more addictive. A faster JS engine will allow adtech companies to send even more junk with every page load. A leaner linux kernel will allow it to run on smaller chips, and we will get IoT crap on more appliances. And so on.
I’m not saying we shouldn’t try to make things more efficient. I’m saying, this could potentially result in more computation (and energy spent), not less.
Yes, the same thing will happen. In the context of resource efficiency, the phenomenon is called the Jevons Paradox. Making a resource’s usage more efficient at a societal level causes more of it to be used, not less. Faster software won’t reduce emissions, it will just make us run more software.
I discuss this in the article (or rather, link to a more extensive article about it: https://pythonspeed.com/articles/software-jevons-paradox/). My initial take: in some cases, yes, but not all.
Induced demand only applies when people actually want something. People want to drive, so when driving becomes cheaper, more people will drive. Only because of the large quantity of people who want to drive somewhere combined with the geometry of the world, this results in congestion never being fixed.
For an example of induced demand being good, look at trains. If you make trains run more often, they may become more popular, as they are now more useful for getting between places. They may become crowded, but they’ll get to their destinations just as fast. And if the trains are getting quite crowded, well, that’s a good argument for running more trains, increasing frequency, which makes the service better yet again, and may induce even more demand. So, the service becomes better with induced demand in one axis (frequency) but not necessarily another (crowding). But there’s another crucial difference: the space savings of trains compared to cars is so great, that it actually is possible to fully meet all demand, so inducing demand until trains are too crowded isn’t inevitable, the way congestion is inevitable with cars.
Finally, we get to efficient programs and computers. More efficient programs probably would induce demand. But how much? It can only happen to the extent that there is demand for running programs currently not met because it’d be too expensive. How large is that extent? It is surely not infinite. How much more efficient can we make our programs? The question of whether we’d use more energy or less depends on the answers to those questions.
I mean, they want to get to their destination, they believe driving is the only valid way to do that, and they reason that more lanes means more ez ride.
It’s frustrating because (here in Chicago) it is absolutely true that trains are late, busses bunch up on routes, and scheduled routes are sometimes too infrequent, but folks who use them as reason not to use transit ignore the indignities they already experience while driving. Oh well.
Am I off topic? At the very least I’m looking too far into the analogy. Pardon me.
Why tho? When do I need a number without any further context?
SEVENis exactly as magic as7, andDAYS_IN_WEEK = SEVENdoesn’t offer anything over assigning directly to the integer while adding the expense of understanding this here thing.Unless we’re just having fun, in which case I’m into it, but I think it’d be nice to indicate it’s a parody library. This does seem to be a good way to achieve dogmatic adherence to the “best practice” of not assigning ints and could fool junior folks into thinking it’s the right move. Teach through humor rather than making folks feel foolish.
Yep.
42is meaningless,ANSWER_TO_THE_GALAXYis not.FOURTY_TWOcontains identical information to the former, except… actually not identical: it’s slower to parse when dealing with these small numbers, especially if English isn’t your first/native language. Maybe on the way high end of the number scale this starts making more sense (ONE_BILLIONis actually somewhat useful maybe?)Maybe — is that a short billion or a long billion?
Steamtinkerlaunch is a wrapper for a bunch of tools supporting Steam games on Linux. It has a bunch of support files but the main script is ~27k lines of bash. I tried digging around in it to fix some issues I was having and the experience made me genuinely unhappy. Ruined my day.
https://github.com/sonic2kk/steamtinkerlaunch/blob/master/steamtinkerlaunch
As soon as you pointed it out, this seemed the obvious best solution. Sort of a workaround for LIST’s inability to reverse the sort order.
Also, if you don’t want the application programmer to have to know that things are stored in reverse order then you can just have some wrapper function that subtracts the user-visible sequence number from u64::max.
It would also display the most recent log entry if I just straight up looked at the bucket in the web UI. Feels pretty intuitive to me.
The initial juxtaposition between the two Hello World’s is not v good. The one that imports the color library outputs colored text, while the one that does not, does not. It shows why you would import something, to get behavior not offered, or not offered in a straightforward manner, by the stdlib. A better example would be
http.serveMuxvsgorilla/mux.Good point! I updated the
Hello, world!example based on this feedback. The HTTP examples are presented later in the post.While it does directly reference Android I omitted the tag, as I believe the meat of this post is adapting a lipo device to direct USB for safe upcycling.
So merry ChristMatz 2022, we got 3.2 where YJIT is stable but off by default. 2023 it 3.3, YJIT’s on by default in a paused state so you can enable it in code, rather than when compiling and passing a flag to executables. No relevant change this year.
Anyone got a sense of when it’ll be on by default? Or remove the flag altogether? Why might someone avoid YJIT, and why can’t we give em an LTS version?
I dunno, seeing all that feature flagging in the low level implementation puts a squirm in my spine. I don’t think it’s like wrong, I just wouldn’t want to be maintaining it. OTOH, that’s all focused and prolly low churn stuff, so I suppose it ain’t hurting no one.
I believe another reason it is still off by default is more people-related: ruby-core is mostly Japanese and C-based. YJIT is maintained by Shopify employees and Rust-based.
Ruby has many of the same problems as Linux – some groups want to move to Rust for safety and speed but the existing maintainers often don’t know Rust at all.
Ruby has improved quite a lot. I love Ruby and used it extensively in the late 2000s. MRI was a disaster. Slow and had memory leak issues in long-running processes. Right now, it’s much faster and more stable. Night and day difference. Leaving Ruby aside, it never ceases to amaze me how good and performant the JVM is. The results in this benchmark are quite stunning.
yea, it’s a great question, and something I thought of mentioning in the article (then forgot).
I think the primary reason it’s still off by default is because of memory overhead. any JIT will usually add a not insignificant amount of memory usage on top of the runtime.
That said, YJIT is really configurable about the amount of memory it will consume. By default I think it’s 48 megs per process max? I know Github uses it but tunes it down a bit to more like 16megs. So possibly in the future it will be on, but by default set to a lower max.
Would be curious to hear from the YJIT team on their thoughts on that!
48MiB is how much executable memory it will use, but it also has to keep track of some metadata which usaly account for 2 or 3x the executable memory. So you’ll easily end up with a 150-200MB overhead with the default settings.
3.4 will have a much more ergonomic memory setting: https://github.com/ruby/ruby/pull/11810
And yes you are right, the main blocker for enabling it by default really is extra memory usage, even though the YJIT team never formally proposed it, but from informal discussion with other Ruby committers, it’s clear that it would be the blocker.
Ah right, thanks for the clarification byroot. Not the first time I’ve thought through that incorrectly - glad to have a clearer setting. Thanks!
I think the main reason to not flip it on by default in 3.3+ was that it could break container / memory-constrained environments if folks upgrade blindly and don’t monitor and increase mem limits appropriately It also can increase startup time for interactive use cases, such as CLIs and such.
I dunno if that was really the right call, but it seems that more conservative approach still holds: I haven’t heard any change in the default YJIT settings for 3.3+.