Threads for 5d22b
-
FeepingCreature
3 days ago
|
link
| on:
Getting meaningful stack traces from Rust tests returning a Result
Could the compiler implement an API on Result that appends each location being error-returned from?
-
One can do that today in a library: e.g., my preferred Rust error library, Snafu: https://docs.rs/snafu/0.7.4/snafu/struct.Location.html.
Edit: I admit that it’s less convenient than having the standard library do this for every
Result, presumably only in debug mode, but the compiler support to build this has been available since Rust 1.46.
-
-
I haven’t watched this, because I’m not a YouTube-watching person, but I thought I’d submit it having mentioned liquid types just a few days ago.
-
One gripe I have with how we talk about “no silver bullet”:
There is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity.
That’s doesn’t happen in any field. When order-of-magnitude improvements happen it’s due to a bunch of different technological innovations that were all in progress for a long time, often interplaying with each other. Even Brooks himself admits this!
A disciplined, consistent effort to develop, propagate, and exploit them should indeed yield an order-of-magnitude improvement. There is no royal road, but there is a road.
-
I forgot about that second part, thanks for the reminder.
I also have an analogy of my own: while we may not have a silver bullet, we can most probably gather enough silver dust to forge one bullet or three. Yes, I think that up to three orders of magnitude improvement, compared to current practices, is attainable at least on principle. For instance:
-
The STEPS project demonstrated the ability to make a home computing system in 20K lines of code, including compilation toolchain, rendering & compositing, desktop publishing, network communication, drawing, spreadsheets… all while traditional systems (GNU/Linux, Windows, MacOS, and their associated software) approach or exceed 200 million lines of code to do the same, or 4 orders of magnitude more. Now STEPS does not include the kernel, but…
-
Casey Muratori pointed out in The Thirty-Million-Lines Problem that while kernels currently (well, in 2015) weighted around 15 million lines (but a fraction of the the aforementioned 200M lines), if hardware vendors got their act together (or I would add were forced by regulation) and agreed on reasonable standard interfaces (ISA) for their hardware and published the damn specs (all those webcams, wifi modules, USB thingies, and of course those fucking graphics cards), we could go back to have small kernels that would only require 20K lines of code or so.
-
Casey Muratori also pointed out that performance really got the short end of a stick, with programs who routinely runs 3 orders of magnitude slower than they could, to the point that users can actually notice it, despite the ludicrous performance of our computers (and I would include the weakest smartphone produced in 2023 in that list). Not only that, but there are low hanging fruits we could pick if we knew how. (A note on refterm being 3 orders of magnitude faster than the Windows terminal: a Windows terminal contributor confirmed to me that those speeds are achievable by the Windows terminal itself, it’s just a ton of work, and they’re working towards approaching it.)
-
Back before my day the Oberon system, by Niklaus Wirth (Pascal/Modula/Oberon languages inventor), was used at his university to do actual secretary and research work for a few years, and the entirety of the OS required no more than 10K lines of code. With a language arguably much less expressive than the STEPS’ languages, on a computer orders of magnitude weaker. This does include the kernel, and the hardware description itself takes up about 1K lines of Verilog.
@jerodsanto, instead of repeating yet again that there is no silver bullet in a way that is most likely to have us abandon all hope, I believe it would help more to highlight that massive improvements, while far from free, are absolutely possible.
-
Casey Muratori pointed out in The Thirty-Million-Lines Problem that while kernels currently (well, in 2015) weighted around 15 million lines (but a fraction of the the aforementioned 200M lines), if hardware vendors got their act together (or I would add were forced by regulation) and agreed on reasonable standard interfaces (ISA) for their hardware and published the damn specs (all those webcams, wifi modules, USB thingies, and of course those fucking graphics cards), we could go back to have small kernels that would only require 20K lines of code or so.
That presupposes that an operating system kernel needs to know everything about the computer’s hardware and serve as an intermediate for all communication with it. That isn’t true, the kernel could limit itself to scheduling and orchestrating access. Then the diverse hardware could be handled by similarly diverse set of separate daemons.
That’s a better approach than making a monolith that has to handle everything and then complaining that everything is too much.
-
I reckon your suggestion has benefits. Done well I expect it will increase reliability and security, probably without even sacrificing performance : computers are so fast nowadays that having components be inches apart makes them distributed systems to begin with.
But it wouldn’t really help the point I was making unfortunately. We need those drivers one way or another. To send you this comment I need at some point to talk to my Wi-Fi module on my laptop. If there are a gazillion different Wi-Fi modules out there we’ll collectively need a gazillion drivers. So what happens in practice is that hardware vendors, instead of giving us the specs, write a (proprietary) driver for the 1 most popular OS (or 2 if we’re lucky), and then let the other OSes to fend for themselves. With a standard ISA for all Wi-Fi modules, we could have one driver per OS, done.
-
-
Casey Muratori pointed out in The Thirty-Million-Lines Problem that while kernels currently (well, in 2015) weighted around 15 million lines (but a fraction of the the aforementioned 200M lines), if hardware vendors got their act together (or I would add were forced by regulation) and agreed on reasonable standard interfaces (ISA) for their hardware and published the damn specs (all those webcams, wifi modules, USB thingies, and of course those fucking graphics cards), we could go back to have small kernels that would only require 20K lines of code or so.
They’re not going to. They are simply never going to do that. Take it as a constraint and move on.
More to the point, this reeks of “We could have a Good, Moral, Simple System if All You Zombies acted right and accepted something that was easier for us to code!” The most successful variant of that basic philosophy was the original 128K Macintosh, and even Apple eventually had to bow to the real world and add enough functionality to keep the IBM PC and clones from steamrollering them utterly.
There’s room to simplify and improve, but I’m allergic to being strait-jacketed into someone else’s idea of a Moral System just so an implementer can brag about how few lines of code they needed.
-
They’re not going to. They are simply never going to do that. Take it as a constraint and move on.
I guess they’re not. Though One argument Casey put there here was that it might actually benefit the first hardware company who does that a great deal. One reason they don’t is because it’s different from the status quo, and different is risky.
There’s room to simplify and improve, but I’m allergic to being strait-jacketed into someone else’s idea of a Moral System just so an implementer can brag about how few lines of code they needed.
Not sure exactly what straightjacket you’re talking about, and how moral it really is. ISAs aren’t all that constraining, they didn’t stop Intel and AMD from updating their processors (they do pay the x86 decoding tax though). Also, the problem is less the complexity of any given ISA, and more the fact that there are so many different ISAs out there. A big driver towards the near insurmountable amount of code in kernels is that you need so many different drivers.
There is one thing however that I would have no qualm writing into law: when we sell computer or electronic hardware, all interfaces must be documented. The entire ISA, all the ways we might modify the operations of the device, everything one might need to use the hardware, up to and including writing a state of the art driver. No specs, no sell. Hardware vendors do write their own drivers, it’s not like they don’t already have their internal specs.
-
-
-
I agree that we’re leaving tons of performance on the table, but I don’t agree with the implication that it’s some kind of moral failing. People optimize towards a goal. When they reach it, they stop optimizing. A purpose built system will always be simple and faster than a general purpose system, but general purpose systems can solve problems not anticipated by people higher up the stack. It’s all a trade off. As Moore’s Law dies off, we’ll see people start optimizing again because they can’t just wait around until hardware gets better, and we know the kinds of things we’re building towards now, so we can be a little less general and a little more purpose built.
-
Actually I’ll do more than imply the moral failing, I’ll outright state it: it is at some point a moral failing, especially when we have a non-trivial number of users, who would like to have less drain on their battery, or would like to wait less time on their software, or would like to suffer fewer bugs and inconveniences. What we might see as wasting a few seconds a day accumulate to much, much more when we have many users.
That, and more performance would allow us to get away with weaker, more economical, more durable, more ecological hardware. The computer and electronics industry is one of the most polluting out there, reducing that pollution (for instance by not expecting users to update their hardware all the time) would be nice.
We might be trading off other things for that performance, but to be honest even there I’m not sure. Writing reasonably performant software doesn’t require much more effort, if at all, than writing crap. I see it every day on the job, we can barely achieve simplicity. If we did that everything would be easier in the long run, including performance.
-
Actually I’ll do more than imply the moral failing, I’ll outright state it
Then perhaps Casey (and you) could direct some ire at the field of game dev. I know Casey and his fans like to promote game dev as the one true field where We Care About Performance™, but as I always point out that’s just not the truth.
And to prove it’s not the truth, I generally just go over to reddit, pull whatever the latest big game release is, and look at a few threads. Apparently right now it’s The Lord of the Rings: Gollum, which benchmarks atrociously even on ultra-high-end gaming hardware, and based on comment threads isn’t really any better on console (which is where the motte-and-bailey argument about game dev often retreats to), suffering degraded graphics in order to keep up any semblance of performance. One thread I glanced at included the savage comment “The Lord of the Rings books had better graphics”.
None of the people involved in pumping out these games seem to think that performance is a moral imperative, or that it’s morally wrong to waste cycles or memory. Yet so often it’s all the rest of us who are supposed to take lessons from them.
So I’ll outright state it: if I took to heart the way game dev does performance, I’d get fired. Casey and friends should spend some time on the beam in their own eyes before complaining about the mote in mine.
-
I know Casey and his fans like to promote game dev as the one true field where We Care About Performance™
His fans perhaps. Casey himself I actually don’t know. He happens to work in the game industry, but he’s on the record about being in it for the technical challenges, not really the game themselves. And if we’re going to scold him about not doing what he’s preaching, we should take a look at what he actually did: the Granny animation system, whatever he did for The Witness… Similarly, I would look at what Mike Acton’s participated in when he worked at Insomniac Games, or what he managed to contribute for Unity.
About games wasting way too many cycles, I know the list is long. Of the top of my head, I personally played Supreme Commander, who I think demanded way more from the hardware than it had any right to, and Elite Dangerous (in VR), whose Odyssey update had unplayable performance for a while, and is still reputedly quite crap.
I would also like to know why Factorio and The Witness take so long to boot. Factorio devs cared a huge deal about performance (and improved it by a couple orders of magnitude over the years), and Jonathan Blow ranted in 2017 about Photoshop boot times, so I guess there must be some explanation? Especially from Blow: I’d like to know why his boot times are somehow acceptable, when Photoshop’s are not.
-
-
-
I can concede that. Computer stuff being really polluting is something I’ve heard I don’t remember where, and trusted then. Maybe I shouldn’t have. I also recall some graphics and numbers on greenhouse gas emissions (from Jean Marc Jancovici), and for these no single sector seemed to dominate any other: we need to reduce everything all around.
I do suspect however the mindset responsible for computer related waste is also responsible for much of the waste pretty much everywhere else as well.
-
-
-
-
-
-
-
It’s such a high bar. “Oh, this tool only doubles your productivity? Thats not an order of magnitude so not a silver bullet”
People really miss the point of the paper. Brooks’ argument was that programmers could make an order of magnitude difference and tools wouldn’t. People joke about 10x programmers but then quote the tool part of the paper.
-
-
you have to understand, at the time the book was written, science fiction and fantasy fandom was not yet mainstream, and realist, detail-oriented taxonomy of every fictional concept was not as far along. it feels silly saying it that way but … I get the feeling it might be hard to imagine what the world was like prior to the cultural victory of a certain segment of fandom, so I’m doing my best to describe what has changed.
this is all by way of saying that it wasn’t intended to be a fully worked-out metaphor. silver bullets are a thing that sounds cool. nobody cared that it’s specifically werewolves they’re usually discussed in relation to. that’s all there was to it.
and yes, software complexity was the obstacle being discussed
-
It’s in the original paper.
Of all the monsters who fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, we seek bullets of silver than can magically lay them to rest.
The familiar software project has something of this character (at least as seen by the non-technical manager), usually innocent and straightforward, but capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet, something to make software costs drop as rapidly as computer hardware costs do.
-
Ah, so I was confused because I hadn’t read the original paper being commented and was only replying to @puffnfresh’s blog post(‘s URL). Serves me right, but I apologize for wasting you all’s time with my confusion!
-
-
The werewolves are… complexity in software?
That always has been my assumption.
-
-
-
-
Disclaimer […] These audits should not be construed as reflecting material safety or security properties of Rust crates.
How should they be construed?
-
-
Also, “just because we’ve audited these against our threat model that doesn’t mean they’re suitable for your threat model”.
Disclaimer: I work adjacent to some of the people working on this but this is me opining in ignorance.
I’d love to see multiple organizations publishing audits like these so other consumers can get a sense of the state of things in aggregate.
-
I’d love to see multiple organizations publishing audits like these so other consumers can get a sense of the state of things in aggregate.
Looking at the
cargo-vetdocumentation, I see https://mozilla.github.io/cargo-vet/importing-audits.html#the-registry.
-
-
-
-
I”m currently looking at wire-free mowers, anyone have any other recommendations? We have a fairly small garden, but it’s pretty complex with various narrow areas and not many straight angles.
So far my main contender is the Ecovacs GOAT, despite the need for navigation antennas.
-
-
-
-
Goats are browsers, they prefer nibbling on shrubs and bushes, and apparently also tend to pull up grass by its roots. Great for clearing brush, but for mowing lawns sheep are better… though I had a friend who grew up on a sheep farm and they are apparently perilously stupid animals. Maybe rabbits? Though that has its own problems. “Sorry for the state of my yard, my lawnmower got eaten by a hawk.”
-
Maybe rabbits? Though that has its own problems. “Sorry for the state of my yard, my lawnmower got eaten by a hawk.”
One can put a large cage over the rabbits and move it around every few hours/days so they mow different areas while being protected. I think I read that in a book. You smart foxes might still get them, though. :-)
-
-
are apparently perilously stupid animals
https://americanliterature.com/author/banjo-paterson/short-story/the-merino-sheep
No idea why this is on an American literature site as Paterson was Australian; but it’s a great yarn (heh).
-
-
-
-
If you want to write code this way, why would you even choose to use Python at all? I wouldn’t use typing for any of these examples. It’s overkill. Just write some tests and spend your energy on something more important.
-
Python has a large and rich ecosystem, and is a popular choice for starting new projects. New projects often eventually grow into large projects, where switching languages would require a significant amount of rewrite. Today, when you just wrote it, remembering whether
find_itemreturns None or throws a KeyError may be easy to keep in your head, but as a system grows and expands (especially to FAANG-scale python codebases), or must be worked on by more than one engineer, strict typing is a must. Almost all Python code at my company is strictly typed, and we have millions and millions of lines of it. It truly makes things much easier in the long run, in exchange for typing out twelve extra characters before you push your commit.-
As a counterpoint, I’ve built a 1MLOC+ codebase from zero in a “FAANG-scale” monorepo. More than half of it was Python without typing. We had no issues static typing would have solved. The arguments in favor of static typing seem worth it on the surface, but in practice you aren’t gonna need it unless it’s critical for performance gains (like Cython or Numba).
-
FWIW, I’ve had the exact opposite experience in the same situation (untyped Python “FAANG-scale” monorepo with millions of lines of code). I think static typing would have helped understand what code meant.
At one point, I wanted to modify a function that took an argument called
image. But what attributes does thatimagehave? It was passed by function after function, and so I spent a bunch of time tracing up the callstack until I found a place where theimagevalue was generated. From there I looked at the attributes that thatimagehad, and went on my merry way…except that the function that I wanted to modify was using duck typing, and there were multiple image types, each with different sets of attributes.If static typing was in use, it’d be obvious what attributes I could access on the image. Less because of compile-time/static checking, but more because it would make it easier to figure out what functions were doing. Yes, this is a problem of documentation, but if I want functions to be documented with what attributes should be passed into them, we might as well do it in a computer-understandable way, so that the computer can check our work.
-
In my experience this is due to expecting to work in dynamic codebases the same as static. In a dynamic codebase I’d put a debugger & enter the repl & see what does the image actually has. This may seems roundabout, but in practice you see more than you can with types because not only can I see the properties, I can play with them & test out calling methods on it with the relevant information.
-
-
This isnt a direct answer, but i think both these question miss the bigger picture of what we want to build towards:
Can we have both a dynamic&inspectable runtime with development tools to catch bugs during development with minimal effort. Type hints are a halfway solution, the future i dream of is a fully inferred static analysis system (idk if types are enough) that can understand the most dynamic code of python/ruby/js/clojure and let us know of potential problems we’ll encounter? Current gradual type systems are too weak in this regard, they don’t understand the flow of code, only “a may have x properties”.
For example:
a = {x: 10} useX(a)
a = {y: 20} useY(a)
Looking at this code, its clear this won’t crash, yet our type systems fail to understand shapes over time. The best we can currently do is {x: number} | {y: number}, which requires a check at each location.
Can we imagine a future where our tools don’t prescribe solutions, but trust you to do what you want & only point out what will fail.
All this being said, this may be bad code, but bad code that works is better than bad code that doesn’t work. This could also enable possibilities we cant dream of.
And then what we traditionally call “pair programming compiler” ala elm, can be lint rules.
-
I mean, Typescript does have an understanding of the execution of code, where it can do type narrowing, and it allows you to write your own type predicates for narrowing types down.
Shapes over time is definitely a thing typescript can do, though it can take a bit of convincing.
-
-
-
-
-
More than half of it was Python without typing. We had no issues static typing would have solved.
Every time I’ve had this discussion with someone it turns out they had tons of issues that static typing would have solved, they just didn’t realize it, or they were paying massive costs elsewhere like velocity or testing.
That said, Python’s type system is kind of shit so I get not liking it.
-
-
We had good test coverage, leveraging techniques like mutation testing (link1 link2). The other half of the codebase used static types and didn’t have a higher level of productivity because of it. Once you have such a large and complex codebase, the fundamental issues become architectural, things like worst-case latency, fault tolerance, service dependency management, observability, etc.
-
-
-
-
Why would I write tests when I have types? :^)
-
Serious answer: types are, among other things, like very concise unit tests (I seem to recall a comment like “Types are a hundred unit tests that the compiler writes for you”, but I can’t find it now), but some bug might still slip through even a strong static algebraic dependent liquid flow quantitative graded modal type system, and tests are another level of defense-in-depth (and I don’t think any language has such a hypothetical type system — I’d like to see the one that does!).
-
I remember seeing someone (I think Hillel) explain that the sensible way to check whether a really complicated static analysis thingy (such as a very complicated static type signature) actually says what you think it says is to try applying it to some obviously-wrong code that it ought to reject and some obviously-right code that it ought to accept.
The idea of unit testing your static types is greatly amusing to me and feels like an obviously good idea.
-
-
Hm must’ve misremembered, sorry. Cheers! :)
-
-
-
(Here are some links for anyone wondering what a strong static algebraic dependent liquid flow quantitative graded modal type system would be.)
-
-
-
-
-
If you’re forced to write Python, perhaps because you have a large existing Python codebase that would be prohibitive to port to another language, setting up a typechecker and using type annotations is an investment of energy that will greatly pay off over time in terms of minimizing bugs. I do agree that it would be better to not write code in Python at all, though, and choose a language that gives you better tools for managing your types from the get-go.
-
Having written an immense amount of Python and uplifting a massive commercial Python 2 codebase to fully type-hinted Python 3: there’s something to be said for being able to drop into a REPL, experiment, mess around, prototype, import your existing modules, mess around some more, and then lean on a static type checker once things are more solidified. It’s a nice workflow for exploratory programming.
-
Yes, I think adding types once an interface has completely solidified and has many dependencies could make sense because at that point it’s mature and you care more about stability than velocity. But starting with type hints when you’re first authoring a file undermines the advantage that Python provides for exploratory programming. That is what I’m against.
-
-
As a more ops-focused person, Python is still the language of choice for a lot of teams I’ve worked in. I’ve used patterns like this when we needed to write code that the team could easily pick up and maintain, but which also needed more explicit safety guarantees than the Python of 5 years ago might be expected to give you.
More concretely, why test for “this never happens” when you can simply make sure it can’t happen and then test that?
-
-
-
ADTs (enums with data) work really well in Rust, and it’s a widely-applicable feature that could easily work in other languages. I’m surprised other languages haven’t added them yet.
-
I think everyone actually have added an equivalent already
- Java has sealed classes
- Python has union types
- TypeScript has union types
- C++ has std::variant and visit
- Kotlin has sealed classes
- Dart has sealed classes
- Swift has Rust enums basically
- C# doesn’t seem to have exhaustiveness checking out of the box, but it does have pattern-matching at least, and you can simulate sealed class with a private constructor.
- Haskell is in the same boat as C# — pattern matching without exhaustiveness
- Go! I think Go genuinely doesn’t have an equivalent feature! Though iirc there are some planes to use interface type-sets to do this.
-
-
Is it named after https://diesel.rs?
-
Seems to be a lot of promises and papers, without a lot of actual description or example code.
-
This is on a GitHub repository. Near the top of the page, above the title “Wyvern: A Language for …”, there should be the text “wyvernlang / wyvern”, in which each word is a link. This identifies the repository. Click on “wyvern” there to go to the front page of the repository. If you’re on mobile, you may need then to find a link “View code” and click it. Then you should be able to see the code. In the code is a directory named “examples”, which contains example code.
-
-
Rust - love it. So safe and blazingly fast, in particular compilation.
In particular I like to have so many dependencies: https://github.com/orhun/rustypaste/blob/master/Cargo.lock Feels cozy.
-
af edited 36 minutes ago | link | flag | reply
Ouch, that’s a truckload of dependencies.
-
https://github.com/orhun/rustypaste/blob/master/Cargo.toml#L26
It doesn’t seem unreasonable. A web server. TLS. Globs, regex, date/time processing.
Note that Rust libraries are typically split into tiny components, so transitively there’s many of them, e.g. cookie parser is a crate instead of being copy pasted code in a web server monolith. It actually helps code reuse.
Complaints about number of deps to me sound like “I can’t eat a pizza cut into 12 slices, that’s too many slices! Cut it in four instead!”
-
-
-
-
-
22:13:31|~/tmp/rustypaste|master✓ λ git rev-parse HEAD f20e2d8d12ceecf65ac60a0acb4b59277b148fac 22:13:36|~/tmp/rustypaste|master✓ λ cargo tree -F shuttle -i axum axum v0.6.18 ├── shuttle-common v0.16.2 │ ├── shuttle-proto v0.16.0 │ │ └── shuttle-runtime v0.16.0 │ │ ├── rustypaste v0.9.0 (/home/matklad/tmp/rustypaste) │ │ └── shuttle-actix-web v0.16.0 │ │ └── rustypaste v0.9.0 (/home/matklad/tmp/rustypaste) │ ├── shuttle-runtime v0.16.0 (*) │ └── shuttle-service v0.16.0 │ ├── shuttle-runtime v0.16.0 (*) │ └── shuttle-static-folder v0.16.0 │ └── rustypaste v0.9.0 (/home/matklad/tmp/rustypaste) └── tonic v0.8.3 ├── opentelemetry-otlp v0.11.0 │ └── shuttle-common v0.16.2 (*) ├── opentelemetry-proto v0.1.0 │ └── opentelemetry-otlp v0.11.0 (*) ├── shuttle-proto v0.16.0 (*) └── shuttle-runtime v0.16.0 (*)
-
-
-
-
-
The number of transitive dependencies is quite important when you consider the attack vectors of a rogue library. To take you analogy back, each slice is cooked by a different chef. Each has a chance to be poisoned, and you eat all the pizza.
Here’s the list of chefs (and their subordinates) you need to trust:
$ cargo tree --format "{r}" | grep -o "https://github\.com/.*"|cut -d" " -f1|sort -u|cut -d/ -f4|sort -u abonander actix alexcrichton Alexhuszagh allan2 allenap Amanieu bancek bitflags bluss bojand briansmith BurntSushi bytecodealliance Canop carllerche chyh1990 contain-rs cryptocorrosion ctz cuviper deprecrated dguo dimbleby djc dropbox dtolnay fizyk20 francesca64 Frommi Geal gyscos hannobraun hsivonen hyperium indiv0 inotify-rs jean-airoldie JelteF jonas-schievink kennytm Kimundi Lokathor magiclen Manishearth marshallpierce matklad mehcode mitsuhiko mvdnes notify-rs nox orhun paholg pyfisch rust-cli RustCrypto rust-itertools rust-lang rust-lang-nursery rustls rust-random rutrum seanmonstar serde-rs SergioBenitez servo smol-rs Soveu srijs Stebalien sunfishcode taiki-e tailhook TedDriggs time-rs tkaitchuck tokio-rs toml-rs unicode-rs vornerYou can be absolutely certain that one of such organizations/users is at reach to be exploited by a malicious state actor.
-
That is true, but you have to consider what are the alternatives. Usually it’s either write the whole thing yourself from scratch, which is highly impractical for most projects, or use dependencies from your OS’s package manager.
Do you know how many maintainers an average distro has? I don’t think any distro does vetting that could catch a state actor. And even good maintainers can’t do much beyond just “LGTM” the tons of code they pull in. And you usually don’t get fewer transitive dependencies, it’s just that package managers using precompiled binaries are worse at showing you the full tree.
In the end transitive dependencies are a form of a web-of-trust, so it’s not a big set of random people. If you’re really at risk of state attacks, you need to review every line of code you use. Rust has cargo-vet and cargo-crev for these.
-
You’re usually better off using the stdlib, and then implementing pieces of what you need. Really the only thing you should ever pull in is a cryptography and hardware abstraction library.
“Web-of-trust” only works where entities are verified in real life, like what GPG does. This is not a web of trust package systems have… it’s a web of “hopefully Im too small to matter to be attacked”.
-
-
-
It doesn’t seem unreasonable. A web server. TLS. Globs, regex, date/time processing.
That by itself isn’t, but you need to look at the actual crates.
Many crates are way too bloated, with unnecessary dependencies, and they’re being pulled in by other crates without concern for efficiency.
It’s a cultural problem. Why is everyone pulling in
serde, whennanoserdealready covers 90% of use cases? Answer: laziness.-
Why is everyone pulling in
serde, whennanoserdealready covers 90% of use cases? Answer: laziness.I’m uncomfortable nowadays with large crate dependency graphs, but I think this is a poor example with an overconfidently uncharitable conclusion.
-
nanoserdemay suffice for an application that works only with the few, concrete serialization formats supported bynanoserde, butserde, as a generic infrastructure for supporting all formats, seems more useful for a library. -
I’m sure many crates have dependencies that could be switched out to reduce bloat, but one would need to remove the heavier dependency from the entire graph, which, for a crate with a large dependency graph, may be difficult for such a standard choice as
serde. If one doesn’t, then using an uncommon choice likenanoserdecould worsen the bloat.
-
-
Attribution to laziness is itself a lazy explanation.
- serde is not just a crate, but an ecosystem. You can serialize almost any data type to any format, because everyone supports serde. OTOH nanoserde is just your structs to/from its couple of built-in formats, and only if you do it manually, because your web framework’s extractors use serde.
- serde is well-documented in both its own docs and on 3rd party QA sites.
- serde is maintained by a well-known respected author, and is very very widely used. It’s in over 45000 crates. nanoserde is in 28 total.
- and in the end, there isn’t that much difference between them in code size. I’ve tried a simple roundtrip of a struct via JSON, and with strip+lto nanoserde gave me essentially the same executable size as serde.
- there isn’t even a practical difference in dependencies. nanoserde’s “no deps” sounds nice, but all of serde’s dependencies are by the same author, they’re just not copypasted into the same folder. Plus they are so commonly used (in 60% of all crates) that they’re de-facto the standard library and you’d have them in any non-trivial project even if you used nanoserde.
So you use less known, less supported, worse documented, less interoperable library, wrangle with your whole dependency tree to avoid pulling in serde anyway, write more glue code, and you save a couple of KB from your executable? So lazy.
-
The problem with bloated dependencies isn’t executable size, it’s compile times.
there isn’t even a practical difference in dependencies.
There’s a big difference. Serde pulls in syn, pro_macro2 and quote. That alone bumps up my clean compile times by 15-20 seconds.
All the points you mentioned are valid, but I doubt that the majority of library authors actually make an informed choice about their dependencies, evaluated alternatives, and measured their compilation times.
Most people follow the herd, and pull in serde, clap, actix without doing their research on alternatives. This is a textbook definition of laziness.
-
-
Just to avoid some misunderstanding here: I’m not calling everyone who uses serde lazy. Obviously there are good reason to use that crate. It’s awesome. I also don’t believe that compile times matter more than the other things you mentioned.
My point is that many, many library authors pull in large dependencies for no net benefit. Compile times are one of the most noticeable examples, because they directly impede my productivity.
I’m talking about cases where they use maybe 1-2% of functionality of the crate. Cases where picking a lighter dependency increases runtime performance, without being brittle or poorly documented. This is what I’m referring to.
I think the way you’ve presented this situation is a false dichotomy, where picking the “standard choice” gives you all these other benefits.
-
-
-
-
-
-
-
-
I think a slightly better proposal was to use a URN namespace (instead of an URI scheme).
In any case, content-addressing is great and should be used more. However, I think there are two issues:
- Large content must be fetched in it’s entirety before it can be authenticated
- No built-in encryption
Number 1 is solved by using a merkle tree (like BitTorrent, IPFS,…). Both are solved by ECRS as implemented in GNUnet or in a slight update I have been working on: ERIS.
-
No built-in encryption
Do you mean that there’s no provision for encrypting a resource while it’s being fetched? It may be rather informal, but it seems to me that section 4 defines a conversion of each
niURI that includes a domain name or IP address into anhttpsURI. A server might not bother serving the resource over HTTPS, but it doesn’t seem to me that this RFC otherwise addresses fetchingniresources much if at all.-
Do you mean that there’s no provision for encrypting a resource while it’s being fetched?
I mean that there’s no enforced encryption at rest or during transport.
As you mention one could use HTTPS or other encrypted transport. But peers that cache the content still can read it and there’s no way to make sure that HTTPS is used.
A nice thing about content-addressing is that it makes the content independent of any location or protocol used to transfer it. Imho, all the more reason to make sure the content is always encrypted.
One could say that these are separate concerns. It is possible to encrypt content and then content-address the encrypted ciphertext. This is the approach taken by IPFS. In practice content on IPFS is not encrypted. Also with BitTorrent, content is not encrypted (causing quite a bit of real harm).
In ECRS and ERIS content is always encrypted and chopped into small blocks. Doing this together results in a readily-usable and network-optimized scheme for content-addressing.
-
-
-
BBC News has a hamburger menu for news topics that, with JavaScript disabled, at least on Android, is simply a fragment link to an unhidden list of topics in the page footer, which I find to work very well, at least for me. (More generally, I appreciate how lightweight the BBC News website feels, at least with JavaScript off.) They also have another hamburger menu for navigating to departments other than News (Sport, Weather, …, Food), which appears to work as a normal hamburger menu even without JavaScript.
-
-
Meanwhile, https://www.ponylang.io has a more normal Material-style hamburger menu that works without JavaScript.
-
-
How does one pronounce this data structure?
-
-
-
SplinterDB uses these under the hood- in the video of the linked Usenix talk, they call it a “beta-epsilon tree”. I have also talked to some folks over at RelationalAI and they called it a “B”-epsilon tree. I think it’s a matter of preference and whatever mental shorthand comes to mind (for those of us with mental narration while reading).
-
-
-
it’s still no match for easy-entry languages like Python or Go. But learning it is a one-time cost. Contrast this with Go, where you may find the apparent simplicity is only skin-deep.
Thanks, I wish people would be more vocal about this. The “hard” part of Rust is learning the language, the hard part of so many other languages is learning the quirks and bad design decisions. You learn the former upfront, you learn the latter from incidents.
-
I don’t know. I have a static site generator that I ported from Python to Go to Rust and now back to Go. The first three were just ways to explore new languages, but the final iteration was “I actually use this, and the Rust version is really hard to maintain” (specifically, despite being more elegant and performant, adding functionality was so much more cumbersome for minimal benefit even as someone who knows Rust pretty well by now).
Additionally, I made more mistakes writing the Rust version than the first Go version despite having the benefit of hindsight and a robust type system. I flubbed some filepath/URL manipulation code in large part because I was so focused on the type system (making decisions about borrowing vs cloning).
In a separate project, I was dealing with a lot of integer IDs which identified different kinds of resources—in order to make sure I used the right ID for the right resource type, I wanted to create an integer newtype (
struct FooID(u64)), but there was a lot of tedium in making all of the necessary arithmetic and conversion traits to have something that was basically usable as an integral type and I still had a couple of frustrations: (1) I still couldn’t automatically instantiate a typed variable from an integer literal a la Go’sx = 0; rather, I had to typex = FooID::new(0)orx = 0.into()or similar and (2) I couldn’t use those arithmetic traits in const contexts, so I had to define a bunch of const functions with similar names. Go just kind of does what I want out of the box—it doesn’t let me implicitly convert between types, it lets me do const arithmetic, and I can instantiate variables from untyped integer constants. If I had to do it again, I’d be tempted just to use u64s for everything (I’ve seen quite a few projects do the same, but I’m not sure which approach is idiomatic), but this is definitely going to yield more bugs.-
-
I’m not sure what you mean. I don’t have many problems with people misusing code. Usually the type annotation makes usage pretty obvious and where it isn’t obvious, I add a small, precise, clear comment. Mostly though I try to keep my interfaces as simple as possible.
I’m not saying Go is perfect; const and Rusty enums would be great, and I would love better compiler optimizations. But Go is just wildly more productive than Rust for most applications in my experience (productivity is not always the most paramount concern either).
-
Yeah, when I wrote some Go, I was missing enums dearly. Also the ? operator; the error handling gets quite wordy. I still think that it’s mostly down to familiarity: Having written Rust code for some time, I can now write stuff in Rust faster than I could in Go, while I allege that someone who’s more fluent in Go will have a contrary experience.
-
I don’t think it’s just fluency. I’m wayyy more versed in Rust than say TypeScript (like nearly 10 years more experienced), but I’m wayyy more productive in TypeScript. I agree that error handling in Go is more verbose than in Rust if you’re not adding context to your errors, but I prefer to add context for debugability (
fmt.Errorf(“doing foo: %w”, err)). You can do this in Rust withanyhowbut that seems like bad practice for library code. Further, I spend a lot of time in Rust crafting error enums and setting up the necessary traits (including convenience traits likeFrom).thiserrorhelps but I still find myself fighting compilation errors in the generated output. Similarly, I also agree that Rust enums would be nice in Go, but unless I’m writing a compiler or something that is just enums all the way down, I just make a struct with a tag field or use interfaces and move on with life (definitely a loss of safety, but I can move so much faster than with Rust that I have plenty of time to test).
-
-
-
-
in order to make sure I used the right ID for the right resource type, I wanted to create an integer newtype (struct FooID(u64)), but there was a lot of tedium in making all of the necessary arithmetic and conversion traits
I don’t want to push you to return to Rust if you didn’t like it, but I think this is a use-case for
ambassador(or delegation, the non-existent built-in equivalent). (Using a procedural macro likeambassadorsurely will not reduce the difference between Go’s and Rust’s compilation speeds, though….)I don’t think I understand why one would want integer newtypes used as IDs to support arithmetic, though.
-
I should note that I still like Rust; I just don’t relate to some of the comparisons with Go. Specifically, Rust is great for systems programming or high performance programming; basically stuff where C and C++ is used today. I also like Rust in theory—the emphasis on zero-cost abstraction and so on.
-
-
Ah, I had been thinking of otherwise meaningless identifiers (such as
mio::Token,id_arena::Id, and a number of types in Salsa).
-
-
-
-
-
christiano
15 days ago
|
link
| on:
What do you use as your current email provider? Why is it better than your previous?
I moved from the free Google ecosystem to the paid Proton ecosystem. I used to pay Google One for extra storage, but Google started to show a lot of ads on its web interface and Android app. I also felt disrespected when I paid for a service and still received tons of advertisements. This was one reason I decided to move away from Google. The second reason is obviously the privacy issue. It is obvious that Proton provides better privacy and gives users full control over their data. Currently, I’m happy with Proton and don’t regret making the decision.
-
Google started to show a lot of ads on its web interface and Android app
I think I never thought of it until reading this, but reading this made me realize that I no longer see ads in Gmail. Testing with an unpaid account on Android, this appears to be a side effect of disabling “Inbox categories”; if I re-enable them, I see ads in the “Promotions” and “Social” category views but still not in “Primary”, “Updates”, or “Forums”.
-
It seems a good solution for those who are still on Gmail and don’t want to see ads.
When I was still using Gmail, I enabled IMAP and installed K-9 Mail on my Android device. It’s a good alternative to the default Gmail app and of course it’s completely ad-free.
-
-
-
I have to assume the name is a setup for registering a domain under a Venezuelan ccTLD…
-
Or maybe it’s short for “yak shaving”.Sorry, just realized that was your point too…(As for the language: I’ve decided I’m not paying attention to languages that make you manually free memory. This is the 2020s, we have flying cars and space colonies, so it should be table stakes for the language to clean up after me.)
-
-
-
The author wants to run a test for each item in a list stored in a JSON file.
Basically, you have two choices to parametrize tests in Rust:
- Code generation
- Custom test harness
They’re both not great.
It seems to me that this article, in reaching immediately for metaprogramming and trying to generate a separate function definition for each list item at compile-time, misses what seems to me the most obvious solution, and maybe the least bad: have a single function that iterates over the list at run-time.
#[test] fn test_from_json() -> Result<(), ...> { let test_cases: Vec<_> = serde_json::from_reader(...)?; for test_case in test_cases { assert_eq!(...); } Ok(()) }If losing the parallelism of having multiple test functions hurts, one can use Rayon to parallelize this.
-
The article mentions the following points, which are good reasons not to run tests in a loop in the same function:
You are however now also responsible for: […]
- Handling of panics in your tests (if there is a panic, it will just crash the test runner unless you handle it)
- supporting all the CLI arguments on cargo test for filtering and listing tests
I’ve personally been frustrated in various testing environments by both of the above points often enough that I now always try to use the test runner to parametrize tests. If that’s not possible, I will often write out the tests by hand rather than use a loop because the ergonomics are that bad.
-
Note that these objections are related to replacing the standard runner.
The OP talks about running cases in a loop in the same function using the standard test runner, so none of these applies exactly, though a variant of point (1) does: the first sub-test to fail will immediately abort the entire test.
-
-
-
You are saying that it’s an entirely different thing, but I’m arguing that it is essentially the same thing. Running all of your tests in a loop is a form of replacing the test runner — with a test runner derived from the original test runner, except that it’s worse because of how it represents and runs individual independent tests.
It’s true that we haven’t literally changed the configuration value of the environment’s designated “test runner”, but, in a sense, it’s more profound because of that: the pattern of running tests in a loop transcends specific test runners, so it’s useful to consider it as a “test runner” itself (or a standard way of transforming an existing test runner into a new one), and then take the analysis with us across disparate testing environments.
The bullet points apply to essentially all reimplementations of test runners, including the transformation of e.g. the Rust standard test runner by running tests in a loop. It’s not an accident that the listed bullet points actually do happen to apply to various degrees.
(I admit that I omitted the first bullet point in my quote because it’s true that the host test runner suppresses output in this case, but it’s still accurate to say that we’ve lost a lot of the host test runner’s functionality related to isolating output on a per-test basis. More often than not, we lose a great deal of the host test runner’s functionality by running tests in a loop, so considering all of the listed bullet points for a new loop-testing implementation is a useful exercise.)
-
-
-
-
If losing the parallelism of having multiple test functions hurts, one can use Rayon to parallelize this.
The problem is not the loss in concurrency, hell the reference (pytest) does not parallelise at all.
But your version has broken:
- error reporting, if the test fails, you just get the information that the test fails at the toplevel, you have to hunt for which subcase
- selection, you can’t select and run specific subcases unless you mess about internally
- interruption, because the cases are not independent you need to catch panics internally or reorganise the entire thing if you want the ability for multiple subcases to fail
pytest handles all of these issues perfectly, every parametrized instance is an entire test instance with its own “name” (for both selection and error reporting), and its own lifecycle (setup / run / teardown), so any given instance failing has no influence on the others.
(3) can be made to work because Edition 2018 allows using results rather than panics so you can accumulate independent failures, but I’d say it’s not really idiomatic (most tests seem to rely on assertions still as their default) and lots of convenience utilities are missing.
-
The problem is not the loss in concurrency, hell the reference (pytest) does not parallelise at all.
and once we start talking about pytest plugins, that problem reappears. and so much more: automatic reruns, performance statistics, etc. turns out there is a lot of benefit in extending the datamodel instead of working around it!
-
Every year, none of my languages, frameworks, or tools are mentioned.
-
-
I wondered whether to use the “ask” tag, but I went with the generic “programming” because that’s what the previous year’s submission did.
A new perspective on this, which largely confirms and reinforces the sad story: https://www.jntrnr.com/why-i-left-rust/
For context: the author of this blog post was the first-named author on a recent overhaul of Rust’s governance and is/was¹ on the Rust Core Team, which was the top of Rust’s government before that overhaul.
(¹ their resignation doesn’t appear to have been processed yet)
The speaker has responded: https://pony.social/@thephd/110446817486174177