I found jujutsu very useful for a local fork of FreeBSD ports, since you can do clean series of commits for each package and then octopus merge them all together for your local ports tree, and rebase all of this continuously.
Based on https://semaphore.io/ it seems like it does a bunch if things around metrics and visualising jobs to help you find slow points and flakey tests.
Where did the name “Swiss Tables” come from? It’s not immediately apparent from OP or the linked Abseil website. (Perhaps it’s in the linked video; I didn’t watch it)
Just to be clear “Carbon” is Carbon Design System, an open source system with workflows, design-to-code, etc. Carbon, the element, drives one of humanity’s hard problems, dealing with the entire Core Carbon Principles, Carbon credits and markets, the Carbon journal, and daily news stories about carbon. In practice, this means searching for solutions to your complex problems will have an extra layer of complexity.
About the only worse name choice would be a short, common word like “go”.
All pointers were represented as pairs of an array and an index; NULL was
simply a pair of NIL and 0. If you cast a pointer to an integer, you got a
cons of the array part and the index part.
Finite-Choice Logic Programming, by Chris Martens, Robert J. Simmons, and Michael Arntzenius. It’s a new alternative to Answer Set Programming that is both powerful and accessible.
A quick search for Dusa (the authors’ implementation language) yielded this surprisingly approachable website with excellent documentation and interactive demos.
Yes, not all defects are obvious. Sometimes, as users we want to make sure we (users and developers) concur there is actually something that needs to be changed.
Here is a recent example where I couldn’t get dotnet test containers working with SE Linux. I noticed that a boolean was default false in the dotnet implementation but true in the java implementation. Now, should I charge ahead, as someone who is barely acquainted with test containers and assume I know best? Or should I ask them hey I found that the java guys are doing that versus we (dotnet) are doing this and start a conversation?
I chose the latter and was pleasantly surprised how quickly the maintainers resolved the issue. Turns out the java implementation is what we want as well.
I like this example because while I thought things should be a certain way, I wouldn’t be surprised if the reaction was negative had I just made a pull request without a conversation.
I just want a standard library solution to replace Poetry and Pyenv. I’ve hit two of the three issues in the article (and many more not listed), but I still use Poetry because it solves the problem. Switching to another non-standard library solution means all my teams now need to learn whatever gotchas those projects have, as well.
I hope that uv could eventually become that. It’s for the most part standards based or adds minor changes on top that could eventually become standards.
As I understand it, much of the frustration from the Linux-distro side over adoption of Rust into Python tools is over platform support: what do you do if your distro supports a platform Rust doesn’t? This feels especially nasty for a distro like Gentoo which is designed entirely around compiling from source on the target machine.
I don’t personally pay that much attention. I just know platform support was the cited reason for a lot of the anger when the Python cryptography package rewrote its compiled extension in Rust.
That, or Tokio’s signal support. In general if your signal handler is more complicated than a single-byte write to a self-pipe (link to my blog post) it’s probably worth revising.
But then you have to think about how to handle those writes on the other end, and that’s where Tokio/async Rust can really help. Here’s a document I wrote about how nextest does quite complex signal handling — apologies for linking to my own post again, but I’ve worked extensively in this area.
That crate seems to at least implement its own method to avoid allocations and use a different register call while performing no println!() ? In any case it comes from a known member of the community/compiler and has had more eyes to it.
It might. If your compute needs are stable, so you are not having to rebuild the boot image all the time. That is probably the limiting factor to using this on a user machine.
If you have to rebuild and reboot every time you want to install and play with a new package and that’s something you do a lot of, I imagine it would get very old, very fast.
It wouldn’t help force unbuffering, but if for some reason you wanted buffering, you could tack | cat onto the end to force it to think it wasn’t going to a TTY, like tail -f log | grep xxx | cat.
What’s wrong with picking Blake2 completely independently from your key round? I mean, why not some KDF and then Blake2? Why does input keying method being password suggest to you that you shouldn’t use it downstream?
(Not that I would reach for Blake2 as my default, but that would be for other reasons.)
What’s wrong with picking Blake2 completely independently from your key round?
What’s wrong with just choosing a random 256-bit key separate from anything?
The implied goal of a password KDF is to derive your key (or IKM if you’re using HKDF afterwards) from the password. And bcrypt simply isn’t built for that purpose.
I mean, why not some KDF and then Blake2?
The implied goal is to get your entropy from the password, and the password hashing algorithm in scope is bcrypt.
If you want a password KDF, scrypt and Argon2 are both password KDFs and password hashing functions. But bcrypt is only a password hashing function, not a password KDF.
The linked article covers this, somewhat redundantly.
Why does input keying method being password suggest to you that you shouldn’t use it downstream?
Let’s say you did some bcrypt + BLAKE2b dance, but don’t want one key, but rather, four keys. Or ten? (It doesn’t matter, really, how many you want. For our use case, it’s enough to be “more than one”.)
Do you run the Password-Based KDF multiple times (one for each key)?
No, because that would be stupid slow on your end (i.e., the defender), but the attackers can crack any of them to learn the password used for all the others. It’s the most wasteful way to solve this problem.
Solution: Only ever do the PBKDF once, and then use the output of that for a faster algorithm (such as HKDF) to split it into multiple derived keys.
The openSUSE project requires spec files to have a license header. It assigns the same license to its RPM spec files as the license of the package itself. So, if a package is GPLv3-licensed, the spec file is also considered GPLv3. The exception is if a package does not have an open-source license, then the spec file is assigned the MIT License.
It seems to be incredibly inconvenient for everybody involved. Is it implied that due to the GPL linking clause, this is (in their opinion) what you have to do?
It’s a really strange choice because the spec file doesn’t link against anything in the code. It’s not to different from a textual description of the build process.
It’s also problematic when the package license changes. Suppose a package with a sole maintainer (who can just do it single-handedly) changes its license from GPLv3 to MIT. Suppose the spec has multiple contributors who all need to agree to any potential license change. Now the spec is stuck with GPLv3 even though the license of the package itself is now much more permissive.
On the flip side, a spec file must (if only to protect maintainers) have some license. “Use the same as the package” is simple to explain, and avoids all kinds of ongoing support requests / discussions / storms in teacups over ‘license changes’ which aren’t really licence changes.
Surely “Use MIT” is even simpler to explains and avoids even more ongoing requests/discussions? I mean what happens if you have 10 contributors to a complicated spec file, then the upstream project changes license from e.g GPL to MIT? You’d need to contact those 10 contributors and get their consent to re-license the spec file! That seems like a wholly unnecessary waste of time.
I can’t say for sure, but it wouldn’t surprise me if “Use MIT” resulted in a steady trickle of well-intentioned-but-ignorant tickets getting raised to the tune of “It looks like you’re relicensing upstreams GPL code to MIT, you can’t do that”.
It would surprise me a lot if that is more work than gathering signatures for everyone who has ever contributed to a spec file any time its upstream relicenses..
We have seen extremely complex projects pull off license changes with no issues… Plenty of the truly huge and complex open source software out there requires contributers to sign a CLA
I think the best argument for it is that if the upstream package maintainers wants to start maintaining the specfile too, i.e. like docker-ce does, having the spec file as the same license as the project makes it easier for the project to pull it into their repo and keep it up to date.
I don’t think that happens very often in practice, but perhaps it happens about as often as upstream license changes, which are also quite rare.
Ive been using Prometheus for as long as I remember for metrics ingestion. It has been rock solid so far, and following the advices for sharding and federation we managed to scale it pretty far.
The issue remains that for longer term storage we’re still not there.
Anybody has a “war story” of trying out Victoria metrics as a replacement ? Or thanos?
We are using Mimir which is similar to Thanos as they are distributed I think. I’d say they are viable options once you hit a scaling limit with Prometheus but that should take a wollte depending on your usecase.
I found jujutsu very useful for a local fork of FreeBSD ports, since you can do clean series of commits for each package and then octopus merge them all together for your local ports tree, and rebase all of this continuously.
What’s the unique feature here?
Based on https://semaphore.io/ it seems like it does a bunch if things around metrics and visualising jobs to help you find slow points and flakey tests.
Where did the name “Swiss Tables” come from? It’s not immediately apparent from OP or the linked Abseil website. (Perhaps it’s in the linked video; I didn’t watch it)
They are called “Swiss” because the team that developed the design was based out of Google’s Zurich office.
https://youtu.be/ncHmEUmJZf4?t=2717
I used to do this too, but occasionally regretted it when a machine rebooted unexpectedly. Instead, I use for a long time now this system: https://leahneukirchen.org/blog/archive/2006/01/keeping-your-home-clean-with-mess.html
Just to be clear “Carbon” is Carbon Design System, an open source system with workflows, design-to-code, etc. Carbon, the element, drives one of humanity’s hard problems, dealing with the entire Core Carbon Principles, Carbon credits and markets, the Carbon journal, and daily news stories about carbon. In practice, this means searching for solutions to your complex problems will have an extra layer of complexity.
About the only worse name choice would be a short, common word like “go”.
My favourite thing about Go is that it collides with “Google Go”, the google search app for emerging markets: https://play.google.com/store/apps/details?id=com.google.android.apps.searchlite&hl=en
To me, “Carbon” is the MacOS API. shrug
Almost seems like Google engineers have an in-joke to make things hard to search for. Maybe they want to keep their search team on their toes.
Luckily the name Gemini was available. Oh wait…
A magit-style Emacs UI for Jujutsu.
I wonder how this compares to
kj::Promiseandkj::PromiseFulfillerPair?Thanks to the ubiquity of mobile phones, it’s very easy to carry powerful RPN calculators and never worry about these things…
Zeta-C ran C on Lisp Machines.
It used fat pointers internally:
The S3 fifo catche eviction algorithm. It’s beautifully simple, but also extremely well presented:
https://dl.acm.org/doi/10.1145/3600006.3613147
Not directly related, but another quite simple idea that’s not the first thing one thinks of: https://lobste.rs/s/zbewy5/load_is_not_what_you_should_balance
From 13 months ago, but I liked reading about the SIEVE cache eviction algorithm https://brooker.co.za/blog/2023/12/15/sieve.html https://cachemon.github.io/SIEVE-website/blog/2023/12/17/sieve-is-simpler-than-lru/
My favourite cache algorithm was the 2Q algorithm: https://www.semanticscholar.org/paper/2Q%3A-A-Low-Overhead-High-Performance-Buffer-Johnson-Shasha/5fa357b43c8351a5d8e7124429e538ad7d687abc
This looks interesting too
Finite-Choice Logic Programming, by Chris Martens, Robert J. Simmons, and Michael Arntzenius. It’s a new alternative to Answer Set Programming that is both powerful and accessible.
A quick search for Dusa (the authors’ implementation language) yielded this surprisingly approachable website with excellent documentation and interactive demos.
Chris Martens gave this great talk at StrangeLoop 2015 Ceptre: A Language for Modeling Generative Interactive Systems
A recent paper on Ceptre, Modeling Game Mechanics with Ceptre
I prefer a good bug report over a hastily contributed fix.
Yes, not all defects are obvious. Sometimes, as users we want to make sure we (users and developers) concur there is actually something that needs to be changed.
Here is a recent example where I couldn’t get dotnet test containers working with SE Linux. I noticed that a boolean was default false in the dotnet implementation but true in the java implementation. Now, should I charge ahead, as someone who is barely acquainted with test containers and assume I know best? Or should I ask them hey I found that the java guys are doing that versus we (dotnet) are doing this and start a conversation?
I chose the latter and was pleasantly surprised how quickly the maintainers resolved the issue. Turns out the java implementation is what we want as well.
I like this example because while I thought things should be a certain way, I wouldn’t be surprised if the reaction was negative had I just made a pull request without a conversation.
https://github.com/testcontainers/testcontainers-dotnet/issues/1312
I just want a standard library solution to replace Poetry and Pyenv. I’ve hit two of the three issues in the article (and many more not listed), but I still use Poetry because it solves the problem. Switching to another non-standard library solution means all my teams now need to learn whatever gotchas those projects have, as well.
I hope that uv could eventually become that. It’s for the most part standards based or adds minor changes on top that could eventually become standards.
I asked the author about uv on Mastodon, he doesn’t look enthusiastic https://piaille.fr/@mgorny@treehouse.systems/113686941213187687 I’m a bit surprised by his response, though.
As I understand it, much of the frustration from the Linux-distro side over adoption of Rust into Python tools is over platform support: what do you do if your distro supports a platform Rust doesn’t? This feels especially nasty for a distro like Gentoo which is designed entirely around compiling from source on the target machine.
Rust is ported to all Gentoo platforms except for alpha and hppa.
there you go
I don’t personally pay that much attention. I just know platform support was the cited reason for a lot of the anger when the Python
cryptographypackage rewrote its compiled extension in Rust.PyPA delenda est.
Yes, it has been very clear for many years that there should be one way to do it. UV is promising. I hope it can become a standard tool.
Would using https://crates.io/crates/signal-hook fix the signal-related CVE?
That, or Tokio’s signal support. In general if your signal handler is more complicated than a single-byte write to a self-pipe (link to my blog post) it’s probably worth revising.
But then you have to think about how to handle those writes on the other end, and that’s where Tokio/async Rust can really help. Here’s a document I wrote about how nextest does quite complex signal handling — apologies for linking to my own post again, but I’ve worked extensively in this area.
That crate seems to at least implement its own method to avoid allocations and use a different register call while performing no println!() ? In any case it comes from a known member of the community/compiler and has had more eyes to it.
Everything on https://github.com/leahneukirchen/leahutils of course. ;)
Since you’ve broken the taboo, my own collection is here :)
http://chriswarbo.net/git/warbo-utilities
would this make any sense to use on a single user system like my laptop?
It might. If your compute needs are stable, so you are not having to rebuild the boot image all the time. That is probably the limiting factor to using this on a user machine.
If you have to rebuild and reboot every time you want to install and play with a new package and that’s something you do a lot of, I imagine it would get very old, very fast.
If the images get too big to load them into ram, using a squashfs/erofs may be an option instead.
Yeah that makes a lot of sense. It might be an interesting thing to try doing !
let us know how it goes!
A useful use of cat, perhaps: avoiding grep block buffering at the start of a pipeline
cat doesn’t change anything in that example.
It wouldn’t help force unbuffering, but if for some reason you wanted buffering, you could tack
| catonto the end to force it to think it wasn’t going to a TTY, liketail -f log | grep xxx | cat.If you want a KDF and have BLAKE2, why would you use bcrypt?
Argon2 is the KDF based on blake2 fwiw, since the article discusses argon and bcrypt.
Because BLAKE2 isn’t a password KDF?
I wouldn’t use bcrypt if I wanted a KDF, but I also wouldn’t use BLAKE2 if I was working with passwords as my input keying material.
What’s wrong with picking Blake2 completely independently from your key round? I mean, why not some KDF and then Blake2? Why does input keying method being password suggest to you that you shouldn’t use it downstream?
(Not that I would reach for Blake2 as my default, but that would be for other reasons.)
What’s wrong with just choosing a random 256-bit key separate from anything?
The implied goal of a password KDF is to derive your key (or IKM if you’re using HKDF afterwards) from the password. And bcrypt simply isn’t built for that purpose.
The implied goal is to get your entropy from the password, and the password hashing algorithm in scope is bcrypt.
If you want a password KDF, scrypt and Argon2 are both password KDFs and password hashing functions. But bcrypt is only a password hashing function, not a password KDF.
The linked article covers this, somewhat redundantly.
Let’s say you did some bcrypt + BLAKE2b dance, but don’t want one key, but rather, four keys. Or ten? (It doesn’t matter, really, how many you want. For our use case, it’s enough to be “more than one”.)
Do you run the Password-Based KDF multiple times (one for each key)?
No, because that would be stupid slow on your end (i.e., the defender), but the attackers can crack any of them to learn the password used for all the others. It’s the most wasteful way to solve this problem.
Solution: Only ever do the PBKDF once, and then use the output of that for a faster algorithm (such as HKDF) to split it into multiple derived keys.
Very curious how the OpenSUSE policy came to be.
It seems to be incredibly inconvenient for everybody involved. Is it implied that due to the GPL linking clause, this is (in their opinion) what you have to do?
It’s a really strange choice because the spec file doesn’t link against anything in the code. It’s not to different from a textual description of the build process.
It’s also problematic when the package license changes. Suppose a package with a sole maintainer (who can just do it single-handedly) changes its license from GPLv3 to MIT. Suppose the spec has multiple contributors who all need to agree to any potential license change. Now the spec is stuck with GPLv3 even though the license of the package itself is now much more permissive.
On the flip side, a spec file must (if only to protect maintainers) have some license. “Use the same as the package” is simple to explain, and avoids all kinds of ongoing support requests / discussions / storms in teacups over ‘license changes’ which aren’t really licence changes.
Surely “Use MIT” is even simpler to explains and avoids even more ongoing requests/discussions? I mean what happens if you have 10 contributors to a complicated spec file, then the upstream project changes license from e.g GPL to MIT? You’d need to contact those 10 contributors and get their consent to re-license the spec file! That seems like a wholly unnecessary waste of time.
I can’t say for sure, but it wouldn’t surprise me if “Use MIT” resulted in a steady trickle of well-intentioned-but-ignorant tickets getting raised to the tune of “It looks like you’re relicensing upstreams GPL code to MIT, you can’t do that”.
To a close approximation, 0 people read spec files, so I’d be surprised at such a trickle.
If I understand correctly, Fedora defaults all .spec files to MIT, and searching around their bugzilla, I can’t find any confused reports like that.
It would surprise me a lot if that is more work than gathering signatures for everyone who has ever contributed to a spec file any time its upstream relicenses..
How complicated a spec file are you getting for a project with few enough contributors to pull off a license change?
We have seen extremely complex projects pull off license changes with no issues… Plenty of the truly huge and complex open source software out there requires contributers to sign a CLA
I assume this is where you impose a CLA on spec changes.
I think the best argument for it is that if the upstream package maintainers wants to start maintaining the specfile too, i.e. like docker-ce does, having the spec file as the same license as the project makes it easier for the project to pull it into their repo and keep it up to date.
I don’t think that happens very often in practice, but perhaps it happens about as often as upstream license changes, which are also quite rare.
Just a guess: .srpm files have a consistent license then.
Congratulations for the team!
Ive been using Prometheus for as long as I remember for metrics ingestion. It has been rock solid so far, and following the advices for sharding and federation we managed to scale it pretty far.
The issue remains that for longer term storage we’re still not there.
Anybody has a “war story” of trying out Victoria metrics as a replacement ? Or thanos?
What do you use for trace data?
I wonder if quickwit could be good for that.
Not a war story, but I moved my personal metrics to VM and it works fine.
We are using Mimir which is similar to Thanos as they are distributed I think. I’d say they are viable options once you hit a scaling limit with Prometheus but that should take a wollte depending on your usecase.
See real stories from users who tried VictoriaMetrics - https://docs.victoriametrics.com/casestudies/