This was my first time doing a public talk. I tried to summarize a pretty big topic in a short time, I hope the details didn’t get too lost. Any feedback would be much appreciated!
Thanks for the talk, it was interesting! Here’s some feedback about how the presentation felt for me:
First section (what are monads and how are they everywhere) was very clear, but a little slow
I don’t understand why algebraic effects are all that desirable yet
I don’t understand how we get from “annotate what errors you throw” to “bundling effects together” or “effect inference”, in fact, I don’t know what bundling effects together means in the context of exceptions, it sounds like effects are actually something quite different from exceptions?
Felt like you were tackling the usability of monads quite a lot, but then didn’t talk about that for algebraic effects beyond IDE support. Also felt kinda notable that the extra features of e.g. Haskell’s do, or LINQ or Elixir’s with expression didn’t come into it, cos they seem like big usability benefits.
In the Q&A you talked about “subtype inference”, which sounds a lot like the structural typing in Typescript and similar to the structural typing in Go and OCaml.
Thanks for the feed back! Those are very valid points.
I didn’t talk that much about the usability of algebraic effects, because right now, there isn’t much. Of course the entire comparison is unfair since Monads are well explored while Algebraic Effects aren’t. And since I wanted to keep this somewhat short and understandable, a lot of cool things about Monads didn’t get mentioned.
Excellent talk, great introduction to monads & algebraic effects!
You touched on the fact that before monads, Haskell used streams for all IO. You mention that this API was quite weird/annoying, and other than your talk I’ve only seen some very high-level descriptions of this online (which I remember coming to roughly the same conclusion). Do you happen to know what actually made this API so weird/annoying?
In the current codebase I am working on, we use RxJS observables for all app state & communication between functions/modules (in a similar pattern to Cycle.js, if you are familiar with that). In this world you have functions that transform source streams into sink streams, and I think this is actually a really nice pattern in my opinion. Dataflow is 100% explicit, ie. you can tell what external resources a function can access just by looking at which streams are passed in as parameters.
Having worked with Haskell previously, I actually prefer this style of IO: it’s much easier to combine multiple input streams from different sources (ie. “read effects”) using stream operators than composing multiple monads (you touched on this with monad transformers).
On your blog, I saw that you mention Pointless, which takes a similar approach: its equivalent of main (output) is a lazy list of commands. Elm also has a similar idea for IO, with ports.
So yeah, wondering if you had any insight on why Haskell moved away from this - are we just re-inventing the wheel badly?
I don’t, or definitely not more than you already have. I’ve never tried this myself, and mostly just repeated what I’ve read in the HOPL paper on Haskell, in section 7: https://dl.acm.org/doi/10.1145/1238844.1238856
Also relevant is the video from the HOPL conference starting at minute 16 or so. There it’s called a “prolonged embarrassment”.
I also read through the relevant sections in the Haskell Report 1.0 and 1.3. They are surprisingly short! But still, the monad version is simpler and shorter.
But I do feel like stream based effects might not have gotten a fair try in early Haskell. The “continuation-based IO” feels like the wrong abstraction. While I was sometimes annoyed by Elm’s ports, they are absolutely practical and very far from a “prolonged embarrassment”. It also feels like these effects as streams fit UI very well. Especially web UIs, where you (mostly) don’t have parallelism.
Agreed on that repeating the meme is just being too lazy to call out which part specifically is broken.
That said, by far the biggest problem I have with Python dependencies is failures during installation of packages that compile C/C++ code that break on wrong headers, libraries, compiler versions etc. Most often with numpy, scipy and similar.
Now, is there hope the situation improves without migrating off of setup.py scripts? I doubt it.
Everything needs to migrate to something declarative/non-Turing-complete so that those edge cases are handled in the tool itself, once and for all packages. I’m not sure if setup.cfg/pyproject.toml cover all the use cases to become a complete setup.py replacement. Probably not (yet).
Yes, that’s the real issue. Django web devs can happily play with pure Python packages all day long and wonder why others are complaining until they try to build a package with C code from source.
Numpy, for example, goes full clown world – last time I checked you needed a Fortran compiler to build it.
God have mercy on your soul if you try to pip install a package with C dependencies on a more niche Linux system (maybe this only applies if you’re not using precompiled binaries?).
I don’t really get this. I can pip install numpy without a Fortran compiler. So can you! It has pre-built binaries, on the Python Package Index, for a bunch of popular operating systems and architectures. I can’t remember the last time I needed to actually build a package from source just to install and use it.
This is not quite inaccurate. Try to install it for example in alpine linux. Or try to install a very new version.
Or use a less common feature not included in the default pre-built packages
’‘pip install’’ does rely on compilers for countless packages.
The packaging ecosystem supports (as of PEP 656) binary packages compiled for distributions using musl. So if you find a package that requires you to install a bunch of compilers on Alpine, it is explicitly not the fault of Python’s packaging tooling – it’s the fault of a package author who didn’t provide a built binary package for musl.
As for “very new version”, numpy already appears to have built binary packages available on PyPI for the as-yet-unreleased Python 3.11.
And if you’re going to customize the build, well, yeah, you need to be able to do the build. I’m not sure how Python is supposed to solve that for you.
Now, is there hope the situation improves without migrating off of setup.py scripts? I doubt it.
No. You already explained in your second paragraph what the problem is. All this talk about poetry or praising package managers of other languages as if the better experience from those wouldn’t heavily rely on the fact that those who use them, do so in a very limited scope.
Remove all the packages that rely on the binary C libraries that do the heavy lifting, et voila, you are at the level of smoothness as other package managers. I would even argue that python virtual environments are unmatched in other languages, if anything.
Which other language provides a high level interface to cuda with a easier set up? Which other language provides functionality equivalent numpy with a smoother setup?
C libraries are installed in different ways in different systems. Pip will try to compile C and this will horribly degrade in the absence of the exact ABI on some dependency the package expects.
I am not really sure this has a solution. Support for different operative systems is important.
You’ve made a couple of points which I don’t necessarily agree with but let me just focus on the original one in this thread.
C libraries are installed in different ways in different systems. Pip will try to compile C and this will horribly degrade in the absence of the exact ABI on some dependency the package expects.
I am not really sure this has a solution. Support for different operative systems is important.
As @ubernostrum has pointed out in a different thread, it’s best if a package comes with a precompiled binary wheel (and statically linked at that). Installation is great then and it’s probably the solution. That said, I can’t install e.g. PyQt5 (i.e. not a unpopular package) on a M1 Mac at the time of writing this. curl https://pypi.org/pypi/PyQt5/json tells me there’s no apposite binary package for it.
The next best thing IMO is to vendor C/C++ library source code (either through a git submodule or even try to download the sources on the fly maybe) and attempt compiling it so that it doesn’t depend on any system-installed headers and libraries. This presupposes a presence of a working C/C++ compiler on the system which I think is acceptable.
What’s fraught with peril though, is to assume that the relevant library (along with its headers) is preinstalled on the system and attempt linking with it. This is just too complicated of a problem to be able to handle all the cases correctly. (1) It relies on the presence of some external means of installing libraries, so it conceptually takes you outside of the Python package manager into apt/brew/etc territory. (2) Your Python project can break when you upgrade the library if you dynamically linked with it, or it can break silently due to an incompatibility if you’re linked statically with it (equally bad). (3) The Python package has no control over the exact version of the system dependency so caveats abound. (4) Your Python project can break if you uninstall the system library, since there’s nothing preventing you from doing that.
If I could change one thing about the the Python ecosystem, it would be to push for the first two solutions to be preferred by package authors.
Yeah, at some level, Python’s deepest packaging problem is that Python performance stinks, so you need to use C-wrappers for math and whatnot, but once you do you run into the fact that however bad Python packaging is, at least it exists, which is more than C/C++ can say.
OTOH, Zig is just one guy, and I haven’t used it personally, but it seems like it can handle a lot of C cross compilation stuff? So ideally, the Python core team should just say “C/C++ is part of our domain, so we’re going to include the equivalent of zig build with CPython to making packaging work.”
Just FYI, Zig has a language creator but it is not “just one guy” any longer. It is being developed with the help of the ZSF, https://ziglang.org/zsf/, which is now paying for core contributions beyond the language creator. There is also a small group of core committers contributing to the language and fixing bugs. It is being very actively developed and their next release will be the “self hosted compiler” (the Zig compiler will itself be written in Zig and built with Zig), which will likely also make it easier for new contributors to join the project. There are also some core contributors beyond the language creator who have started to specialize on Zig’s use cases as a x-platform C compiler, for example by focusing on areas like libc improvements (shorthand: zlibc) and on the various target platforms beyond Linux, like macOS, Windows, WebAsm, embedded, etc. It’s likely true today that Zig is the world’s best x-platform C build toolchain, and that will likely only get better and better over time due to the vitality, pace, and mission of the open source project (especially compared to the alternatives).
Your idea about how Zig could help the Python C source build process is a good one. There is a nice post here about maintaining C code and C cross-compilation issues with Zig:
Unfortunately some Python code relies on C++, Fortran, and other exotic languages to build sources, since the Python C extension module interface is so open and so many other languages can expose something that speaks the C ABI. But you’re right that C modules written in C are most often the culprit.
The creator of the language also talked about some of the technical details behind zig cc here:
OTOH, Zig is just one guy, and I haven’t used it personally, but it seems like it can handle a lot of C cross compilation stuff? So ideally, the Python core team should just say “C/C++ is part of our domain, so we’re going to include the equivalent of zig build with CPython to making packaging work.”
I think that’s a fantastic way to improve the Python packaging status quo 1, 2! Zig is a game changer when it comes to portability.
A Rust compiler must also implement unsafe, of course, which disables a lot of the checks that the compiler makes.
This is just wrong, and it’s frustrating to hear it repeated so often. unsafe permits calling other unsafe code, dereferencing raw pointers, reinterpreting unions, and implementing unsafe traits like Send and Sync. It does not “turn off” the type system, borrow checker, or anything else.
While I understand this isn’t the most technically advanced article out there, all I’m trying to do is provide the same learning moments I had to other people in similar positions to me.
Just to add to what @Loup-Vaillant said above, one of the things Rust is sorely lacking is alternative learning paths, driven by the experience of people from various backgrounds. Rust is so large and intimidating that very few people choose to write about it in a manner that’s not just rehashing official documentation with some advocacy sprinkled on top. Your post involves a bunch of cool idioms related to a very practical problem. It’s a very good post that fills a very relevant niche, not just with PLT in general, but also with Rust in particular!
I object-level agree that the Result type is cool - I thought precisely the same thing a decade ago when I learned about Algebraic Data Types and specifically the Maybe and Either types in Haskell. That isn’t a joke, I really was blown away by these programming constructs in Haskell when I first learned about them there, and I was surprised that no programming language I had ever previously used had support for what were obviously such good ideas. I still think that one of the most exciting things about Rust is that it popularized ADTs in a mainstream general-purpose programming language (not to say that Haskell is obscure, exactly, but Rust is probably more widely used than Haskell at this point).
So I’m inclined to agree that Rust’s Result type shouldn’t be conceptualized or taught to beginners as some special Rust thing, but rather as a (genuinely quite generally useful) application of a well-established programming language construct that Rust happens to make use of. And it’s definitely worth questioning why it took so long for the ability to create an Maybe-like ADT to become a mainstream programming tool, and to wonder about what other broadly useful programming conceptual tools exist, but just aren’t available in the mainstream programming languages that a lot of the world’s software gets written in.
Either (and Haskell data types in general) are just tagged unions, which plenty of languages have. You could argue that inheritance based polymorphism is the OO language equivalent.
Obviously every older imperative language has Maybe as well in the form of pointers - the problem of course is that all pointers are always maybe and there’s no mechanism to ensure you check before using :)
I’ve recently started using a library for C# called OneOf, which makes it easy to implement Result and Option types, and it’s really cleaned up my error handling, especially solving the “never return null when the caller expects an object” trap that C# and Java dump you in by default. There’s little/no syntactic sugar in the language to help with it, but as a lisper, it doesn’t really bother me that match is a method rather than a keyword.
To be honest, I never understood what “cool” means, except as something that kids say, so no, I never used the term, as a kid of otherwise. But when something is described as cool, it’s a very good litmus test for me signalling it’s something I don’t need to pay any attention to.
Think twice before posting again. Is your contribution useful and kind? If not, refrain from expressing yourself. No one is getting paid here. You get what you give,
rustfmt also provides many options to customize the style, but the style guide defines the defaults, and most projects use those defaults.
I’d be curious if the team thinks this gives a positive or negative experience to users.
One of the few unquestionably good experiences I’ve had working with a large Go project is that gofmt precludes all arguments about style. rustfmt has knobs for days, and I wonder how much turning they get.
The bigger difference between rustfmt and gofmt are not the knobs, but the fact that gofmt generally leaves new line placement to the programmer, while rustfmt completely disregards existing layout of code.
OTOH to me rustfmt’s “canonicalization” is a destructive force. There are many issues with it. One example: I strive to have semantically meaningful split of code into happy paths and error handling:
let result = try.to().do.something()
.or_else(report_failure).thx().bye?;
To me this gives clarity which code is part of the core algorithm doing the work, and which part is the necessary noise handling failures.
But with rustfmt this is pointlessly blended together in to a spaghetti with weirdo layout like this:
let result =
try.to().do.something().or_else(report_failure).thx().bye?;
or turned into a vertical sprawl without any logical grouping:
let result = try
.to()
.do
.something()
.or_else(report_failure)
.thx()
.bye?;
rustfmt doesn’t understand the code, only tokens. So it destroys human input in the name of primitive heuristics devoid of common sense.
I want to emphasise that rustfmt has a major design flaw, and it’s not merely “I don’t like tabs/spaces” nonsense. The bikeshed discussions about formatting have burned people so badly, that it has become a taboo to speak about code formatting at all, and we can’t even criticise formatting tools when they do a poor job.
As a user, I like that rustfmt has some knobs, it lets me adjust the coding style to use within my own project, so it’s internally consistent.
What matters to me, as a developer, is that the coding style within a project is consistent. I don’t care about different projects using different styles, as long as those are also internally consistent, and have a way to reformat to that style, it’s all good. I care little about having One True Style. I care about having a consistent style within a project. Rustfmt lets me have the consistent internal style, without forcing the exact same style on everyone.
It’s worth saying that Rust is targeting C and C++ programmers more than Go is, and at least in my experience it’s those programmers who have the most idiosyncratic and strongest opinions of what constitutes “good style.” “My way or the highway” may not work well with many of them.
Happy that this finally shipped. I almost unsubscribed from the feature request discussion for this, even though I wanted to keep up with the status. The team ended up delayed shipping this feature and people are apparently unable to refrain from asking for status updates.
Yeah the constant “hey when is this happening” questions were unhelpful. The team did give periodic updates on the delays, along with confirmation that it was being actively worked on, and estimates of when it may be done, which is more than a lot of organizations give for new features.
This a cool example of the severe limitations of microbenchmarking. In microbenchmarks, the performance impact of the extra function call is present, but it completely disappears in larger benchmarks due to JIT optimization. This is also good evidence that benchmarking should be done in the context of real applications and with a focus on what’s actually being observed as slow on real workloads.
The description of how to make a trampoline in Rust to translate from the Go calling convention to the Rust calling convention and back is great! Also a cool example of why the asm macro and naked function attribute are so useful!
Doing the main work for my Big Data final project, which for me means some basic analysis of Google’s Open Source Insights dataset to understand the relationships between project popularity and vulnerability rates and severity. Should be fun!
An amazing history of an amazing set of tools, with a finishing line where I realize I’ve been mispronouncing it for years. Cheers to Valgrind (pronounced correctly now), and cheers to Julian, Nick, and everyone else involved in making it all work!
This is sneakily an introduction to the value of recursion schemes, which are fantastic.
For the unfamiliar, recursion schemes are a way to describe patterns of recursion separately from the operation to do at each recursive step. In the final version of this code, generate is the recursion scheme, and generate_from_boxed is the operation to do at each step.
I think part of why recursion schemes have had difficulty breaking through is that they’re named terribly. “Catamorphism” and “anamorphism” are the two basic ones, and the names only get worse from there. Seeing something like this makes me hopeful about the possibility of recursion schemes gaining wider adoption by abandoning the old way of teaching them!
I think terrible naming and other communication issues hold back just about everything in the PLT world. Cue the old joke about “Monad is just a monoid in the category of endofunctors”. I certainly why some people perceive PLT folks as having a potent “ivory tower” culture.
Yeah, this is part of why I’m glad Rust didn’t start out with support for higher kinded types. If it had them from day one, people likely would have directly imported Monad, Applicative, Functor, et al from the start, tying Rust to these terms which (imo) ought to be left behind.
I agree with this, Monad etc are heavily loaded terms at this point. I think a more effective way to motivate people is to show the benefits of these abstractions well before delving into the theory.
If we leave these terms behind, then who will write the dozens of Rust-oriented (as opposed to Haskell-oriented) articles trying (and usually failing) to explain what a Monad is? /s
More seriously, naming is hard. And I’m sure there would be a lot of pushback from not naming things as they are in the most popular functional languages.
What’s a CS degree like these days, anyway? Back when I was in school decades ago, I don’t recall us covering much if any theory of programming. We just jumped right into structured programming, data structures and all that.
And I’m sure there would be a lot of pushback from not naming things as they are in the most popular functional languages.
On the contrary, I think for the right language will bring the functor/applicative/monad abstractions to the mainstream, probably not under those names, however. Angry people in comment sections/social media are not the whole world, thankfully.
This sentiment is maybe eight decades too late. The words “functor” and “monad” come from philosophy, borrowed from Carnap and Leibniz respectively. The word “category” is also borrowed, from Plato and Aristotle. You can suggest new words if you like, but perhaps it’s worth considering why the original category theorists borrowed existing words instead of making up new ones.
The idea of “the right language” suggests that you’re thinking exclusively of endofunctors, internal monads, and other internal structures within a single programming language. However, it’s important to recall that every language has its own category of types (via native type theory), and so we have functors from one language to another. This abstraction is not language-specific and has been studied for half a century under the name “functor”. Here we could have another nomenclature bikeshed; surely we should call these “compilers” or “translators” instead.
Depending on the situation, I would call them flat_map or similar. I would teach them as something like “say you have a tree of computations: if values in the tree can generate new nodes in the tree, then that’s a special case that you need to think about”.
I think that your comment, as is, is baseless: it makes a claim, which is fine, but then it tries to support it by example which is not convincing. You take a sentence of category theory as an example of communication issue in Programming Language Theory. The sentence is appropriate for audiences in some context (… people interested in category theory) and not in others; it is not in itself a communication issue, it all depends on the context in which it is used.
There are many interesting things to be said about the interplay between various communities (academic and non-academic ones in particular) on programming language design and implementation. But that requires nuance and going above tropes such as the ivory tower. Repeating it without providing any such nuance does not improve the discussion in my opinion, it is impoverishing.
P.S.: this being said, I totally agree that zylomorphisms, zygomorphisms, patamorphisms etc. are impossible names, they sound satirical to me and should probably be avoided. (Histomorphism is somehow slightly better)
I think that your comment, as is, is baseless: it makes a claim, which is fine, but then it tries to support it by example which is not convincing. You take a sentence of category theory as an example of communication issue in Programming Language Theory. The sentence is appropriate for audiences in some context (… people interested in category theory) and not in others; it is not in itself a communication issue, it all depends on the context in which it is used.
Yes, that’s the point of the joke (and I did explicitly refer to it as a joke). People in the PLT world try to explain concepts to outsiders in terms that are only meaningful to insiders. It’s a widely-known and relatable joke among programmers, suggesting that there may be a communication problem. You’re free to disagree, of course–I certainly can’t prove it one way or the other, nor am I interested in trying.
There are many interesting things to be said about the interplay between various communities (academic and non-academic ones in particular) on programming language design and implementation. But that requires nuance and going above tropes such as the ivory tower. Repeating it without providing any such nuance does not improve the discussion in my opinion, it is impoverishing.
I was noting communication problems impede PLT ideas from achieving adoption in industry, I wasn’t attempting to examine interplay between communities. I may still have failed at my goal, but I don’t see the point in telling someone they failed at something they weren’t attempting in the first place. 🙃 🙂
It’s a widely-known and relatable joke among programmers, suggesting that there may be a communication problem.
So a large group (programmer) widely spread a joke meant to criticize a subgroup (programming language theory people), and from this you conclude that the subgroup has a communication problem. Many other conclusions would seem reasonable, including that members of the larger group are prejudiced against the subgroup or dismissive. Explaining this joke away as a “communication problem” from the subgroup seems to ascribe fault in a very one-sided way in this social interaction, when I think again that the situation requires more nuance. (For example: some PLT people do come off as elitist or condescending based on their attitude, and in some cases their choice of words, but there is also a complex relation between programmers and formal/theoretical programming education, combined with an unhealthy tendency to expect wonders from some ideas (including category theory) with hype-disappointment cycles, etc.).
I was noting communication problems impede PLT ideas from achieving adoption in industry, I wasn’t attempting to examine interplay between communities.
One community: PLT people communicating on their work. Another community: programmers in the industry or free software projects.
Thank you! That was my goal writing this, to motivate people to learn recursion schemes in a way that felt like it naturally falls out of building a cache-local evaluator for recursive expressions. I was delighted to find out that it does naturally fall out of building a recursive evaluator in another idiom, which in retrospect makes sense - of course there would be deep connections between different ways of expressing recursion.
My knowledge of Greek teaches me that “anamorphism” and “catamorphism” mean “up shaping” and “down shaping”. I get that there are a lot of cases of simple Anglo-Saxon terms being turned into jargon made up from Latin/Greek stems, but “up/down shaping” (or up/down folding) seems like it’s just as good as a jargon term. Like in literature, “katabasis” is used to distinguish a character going down into the metaphorical netherworld vs. just going down the stairs, but I think for category theory, “down folding” has enough context that it doesn’t need a Greek term.
I’m not sure I understand your complaint. Is the issue that the “old way of teaching them” is not clear, or is the issue that “they’re named terribly”?
You’re right that many languages offer fold, and fewer also offer unfold. My issue is that the space of recursion schemes is much richer than these, but the obtuseness of the existing terminology and lack of good educational material which sidesteps it has limited the spread of recursion schemes as a concept beyond the fold/unfold functions.
I see. For what it’s worth, I think that the issue is that the typical popular programming language was not designed, but copied from previous languages. When a language is designed from scratch, there is an opportunity to improve, but most languages do not get that opportunity.
For example, one of the recursion-schemes authors worked on C♯, but rather than a deeply-integrated object protocol which adds folds and unfolds to each collection, they developed LINQ. (They did also add folds to every collection with an extension method, based on the ability to enumerate it; this is similar to how Haskell’s prelude changed to allow all collections to be Foldable.)
A real problem with string_view is that by design it isn’t memory safe: you can return one from a function, store one in a strict, or whatever and it will silently carry a pointer to its backing store, whether or not the backing store is kept alive.
Personally I think it’s a foot gun of an API given how easy it is to use it in a way that results in UAF bugs.
This feels slightly unfair to call out string_view, iterators have all the same problems, do you consider them a foot gun of an api too? It is a foot gun, but it’s not any more of a foot gun than char * and maybe only slightly more of a foot gun than string&.
And it at least has the advantage of consistently being a non-owning reference to the backing store, something that could go either way with char *.
In the context of C and C++, string_view is an improvement over alternatives. However, Rust has shown the value of statically guaranteeing no dangling pointers. Rust’s &str (the equivalent of C++‘s string_view) can’t point to a string which is no longer valid to reference. That’s pretty great!
Sure. And if the original post had said something like “it’s too bad C++ doesn’t allow string_view to be memory safe by design.” I probably wouldn’t have made an objection. This is a “real problem” with vast swaths of C++ apis though, it is not notable or exceptional that string_view suffers from it.
Sure, in the context of C++, it’s not notable that string_view has this problem, but in the broader context of this set of competing languages (C, C++, Rust), it’s worth noting when a new API is offering less than state of the art guarantees.
But by focusing on the api the implication is the specific api is flawed as opposed to the language. No new C++ api will ever be “state of the art” without a language change. We’re not talking about adding C++’s string_view to Rust where it would make sense to criticize the api for not being memory safe, where the option to do better exists.
I’m curious about how this works. If in Rust, I have a reference counted object A, that points to a reference-counted object B, that has a string field, and I use &str to take a reference to a substring of that string field. I then pass A and the substring to a function, which follows the reference from A to B and truncates the string, how does Rust enforce the bounds checks? If I completely invalidate the range that the substring covers, what happens when I try to use it?
I believe the answer is that the Rust borrow checker would prevent objects reachable from A from being mutated during the function that has the substring reference and so I can’t express the above idiom at all. In C++, there is no such restriction and you get all of the power, flexibility, and the footguns that accompany this.
I think (and could local Rust experts please correct me if I am wrong; @matklad, @kornel) Rust will not let you both have a mut reference from A to B and an &str to substring in B since that would mean you have two references and one of them is mut. You could have a non-mut reference from A to B and an &str substring to B but that means you wouldn’t be able to modify B in your function. But I could be complete wrong.
Yeah, this is mostly correct, with the asterisk that a) what Rust really tracks is aliasing, rather than mutability, and it is possible to use interior mutability to mutate aliased data and b) you can use unsafe and raw pointers to gain all power, flexibility, and even more footguns.
I consider atring_view to be a new API introduced well into the period of knowing that dangling pointers are bad. I couple that with it behaving as a value object rather than iterators that have pointer semantics in my belief that string_view is bad.
Iterators date to the earliest days of c++ and have a well understood pile of issues that make them not memory safe (the for(:) using iterators removes a lot of safety checks and introduce vulnerabilities from previously safe code with indices)
Modern C++ APIs are not designed to eliminate memory safety bugs, they are designed to permit coding styles that eliminate pointer safety bugs. Combined with the lifetime annotation on accessors, a compiler can warn if you have locally created a string view that outlives the object from which it was created. A coding convention that says not to capture string views (pass them down the stack only) then gives you most of the lifetime safety that you need.
The corner case is if the object that the string view references is reachable from another object and the called code mutates it in such a way that it invalidates the string view. For string views derived from something like std::string, this could be avoided by having the string view reference the string object and its bounds but that adds performance overhead and would reduce adoption and it cannot work with string views derived from C strings: One of the big benefits that you get from string views is the ability to provide safe implementations of APIs that are exposed to C and take C string arguments and can also be used from C++ with string objects.
But new APIs should be designed such that they make it very easy - to the extent I’ve seen multiple blogposts and mailing list comments on the memory unsafely of string_view when used in a way that looks reasonable. If a new object is added that is trivially unsafe if moved, copied, or otherwise stored is introduced, then that object should not support copy or move.
One of the more common errors in llvm is storing the semantically equivalent StringRef, which often works fine due to other objects keeping buffers live long enough. Except when they don’t, and then you get crashes or worse not crashes but whatever happens to be present at the time.
It is not reasonable to add new APIs to C++ where code that looks correct is not memory safe.
But new APIs should be designed such that they make it very easy - to the extent I’ve seen multiple blogposts and mailing list comments on the memory unsafely of string_view when used in a way that looks reasonable. If a new object is added that is trivially unsafe if moved, copied, or otherwise stored is introduced, then that object should not support copy or move.
If you remove the copy constructor in string_view then you eliminate its only valid use: passing down value type down the stack to other functions for temporary delegation. The problem is that C++ doesn’t differentiate between copy and capture, so there is no way of implementing a thing that can be passed down but cannot be captured. You could force it to be passed by reference, but then you’re adding an extra indirection for a two-word object and that will hurt performance.
The main reason that string_view exists is to have an explicitly non-owning type that replaces passing a char* and a size_t in a way that makes it easy for one of them to be lost (for other code to assume that the char* is null terminated, for example).
As you say, llvm’s StringRef is analogous and any code that captures a StringRef is a bad code smell and gets caught in auditing. Prior to StringRef, code either defensively allocated std::strings all over the place and hurt performance or decided that performance was important and captured const char*s. StringRef has been a huge improvement over the prior state.
If you have a proposal for a performant, memory-safe, non-owning string reference type that could be implemented in C++ (even with some modest language additions, but without completely redesigning the language) then I’d be very interested to hear it because it’s something that I’d happily adopt in a bunch of codebases.
No, if you remove copy you force pass by reference.
That has the benefit of meaning you can’t make a string_view outlast its lexical context, and you restrict it to what it ostensibly is: an abstraction API over the many variations on what a string is. In exchange for having to put an ampersand in you remove - or at least significantly reduce the footgun-ness of the API and potentially improve perf+code size as you can pass a pointer/reference in a single register :D
No, if you remove copy you force pass by reference.
But you also allow capture by reference for lambdas, for example, so you’re still not memory safe.
That has the benefit of meaning you can’t make a string_view outlast its lexical context
True, but that just moves the problem. You can trivially make the reference outlive its lexical context, for example by lambda capture, by using std::forward_as_tuple to capture a parameter pack, and so on. Worse, the things that capture now look a lot more like things that are safe to capture and so it’s harder to spot in code review.
You also introduce a load of more subtle lifetime issues. By passing std::string_view by value, you can construct them in arguments from things like const char * or std::string references. This makes it very easy to gradually transition an API from one that takes const std::string& and const char * to something uniform. If std::string_view had to be passed by reference then creating a temporary std::string_view and passing it would be very dangerous because the lifetime of the temporary could very easily be shorter than the use, even in the common case where the underlying string had a longer lifetime. Auditing code for this would be much harder than spotting misuse of std::string_view.
If you want a safe string view, then it needs to be integrated with string memory management and prevent any mutation of a string that has outstanding string views. That would be impossible to integrate with APIs that exposed const char* parameters and so would not have the incremental migration benefits of the current design.
So is your argument string_view just shouldn’t exist? “by design” implies to me you think there is an alternative design for string_view that would be memory safe.
I couple that with it behaving as a value object rather than iterators that have pointer semantics in my belief that string_view is bad.
string_view is tantamount to a pair of iterators, I’m not sure how that makes it a value object and iterators have pointer semantics.
My argument is that the existent of bad APIs in the past should not be used as a justification to add new unsafe APIs.
The fact that the implementation is/may be two pointers is irrelevant, it could also be a copy of the data, it could be a pointer and a length, it could be a unordered_map from size_t char, the implementation is entirely moot.
The string_view API behaves like a value type - it looks (in use) to be a memory safe value type like std::string.
The iterator API is via the standard pointer operators *, ->, etc. All things that display very clearly that they are semantically only pointers to data, nothing else.
What matters is how a type behaves, not your thoughts about how it should be implemented.
Your argument is “C++ is filled with things that are unnecessarily unsafe because of backwards compatibility, therefore it is ok to add new unsafe things”, which is not compelling to me.
Your argument is “C++ is filled with things that are unnecessarily unsafe because of backwards compatibility, therefore it is ok to add new unsafe things”, which is not compelling to me.
My argument is it is impossible to add a memory safe string_view to C++.
These things aren’t “unnecessarily” unsafe, they’re intrinsically unsafe.
The argument from a memory safety standpoint is “just don’t use C++”, not “don’t add things to C++”. Adding string_view does not make C++ less safe.
string_view can’t be a copy of the data because then it ceases to be a view, you would just use string then.
It is? How do you implement string_view::substr? I suppose it could be substr(pos, n, LambdaTakingConstRefStrinView f) -> decltype(f(declval<const string_view&>()))
And
auto s = strdup(“ok”);
string_view v(s);
free(s);
f(v);
is still obviously a UAF even with a non-copyable non-movable string_view.
Though I suppose to solve that problem we could again use the same continuation trick, and not make the char* ctor public:
Yes you can explicitly free the backing store but there is a world of difference between safety in the face of explicitly releasing backing buffer - what you’re suggesting is not that far from shared_ptr<std::yolo> ptr = …; ptr->~yolo();
The specific issue I have with string_view is the ease with which it makes UAFs possible, without apparently doing anything sus.
I think I said elsewhere, but LLVM has llvm::StringRef which has the same API semantics, and it is a frequent source of memory bugs - only sometimes caught during review. It has the same issues: it often appears to work due to incidental lifetime of the backing store, or the general “releasing memory doesn’t magically make the memory cease to exist” behavior that makes manual memory management so fun :)
Sigh, why does lobsters occasionally hold off multiple days before it says “there are replies to your compelling commentary!” and make me seem obsessive.
This criticism totally depends on what you use instead! If the alternative is a pointer or reference, they obviously have exactly the same problem – they can outlive the memory:
use_after_free(std::to_string(42).c_str());
I’ve heard this criticism before, but I don’t think it’s relevant with the right mindset:
It isn’t a string replacement! As a slice data type, it’s just a fancy reference. It exists to replace references and pointers, not owned strings.
As the article mentions, a good use is function arguments. The reason is that the caller only needs to make the memory outlive the call. This may seem like a problem if the function needs to store the string, but that’s not a (valid) use case, as then, it needs an owned string, and should therefore take an owned string.
It is as a unifying replacement for other pointer-ish interfaces that it really shines:
C++14:
void foo(const std::string&);
void foo(const char* c_str);
void foo(const char* ptr, size_t len);
void multifoo( // clang-format force line break
const char* a_ptr, size_t a_len, // clang-format force line break
const char* b_ptr, size_t b_len, // clang-format force line break
const char* c_ptr, size_t c_len);
C++17:
void foo(std::string_view);
void multifoo(std::string_view a, std::string_view b, std::string_view c);
For any Rustaceans, I built woah for exactly the need this article lays out. It provides a type woah::Result<T, L, F>, where L is a “local error” you can handle, and F is a fatal error you can’t. woah’s Result type propagates fatal errors up with the ? operator, giving you a std::result::Result<T, L> to handle local errors.
It’s pre-1.0 until the Try trait is stabilized, but can be used on stable if you’re willing to forgo the ? operator and do the propagation manually.
Managing multiple different levels of errors is still difficult for me in Rust. I’ll take a look at the crate but I’m wondering if you can explain what the advantage is of using that over Result<T, LayerError> where
The difference is in having separation between the “local errors” and “fatal errors” and how that interacts with the ? operator.
woah::Result is equivalent to Result<Result<T, L>, F> in terms of how it works with the ? operator. Take the following example, copied from the project’s README.md:
use woah::prelude::*;
use rand::prelude::*;
fn main() {
match get_data() {
Success(data) => println!("{}", data),
LocalErr(e) => eprintln!("error: {:?}", e),
FatalErr(e) => eprintln!("error: {:?}", e),
}
}
/// Get data from an HTTP API.
fn get_data() -> Result<String, LocalError, FatalError> {
match do_http_request()? {
Ok(data) => Success(data),
Err(e) => {
eprintln!("error: {:?}... retrying", e);
get_data()
}
}
}
/// Make an HTTP request.
///
/// This is simulated with randomly returning either a time out
/// or a request failure.
fn do_http_request() -> Result<String, LocalError, FatalError> {
if random() {
LocalErr(LocalError::RequestTimedOut)
} else {
FatalErr(FatalError::RequestFailed)
}
}
/// Errors which can be handled.
#[derive(Debug)]
enum LocalError {
RequestTimedOut,
}
/// Errors which can't be handled.
#[derive(Debug)]
enum FatalError {
RequestFailed,
}
The key line is the match do_http_request()?. The ? operator will pass any FatalError up to the caller of get_data, and return a std::result::Result<String, LocalError> which is what get_data is matching on. In the error case on that match, we have a RequestTimedOut, so we can retry.
I highly recommend anyone interested in this article also read the paper “Build Systems a la Carte”. You can skip/ignore the Haskell bits, and focus on the description of build systems as a combination of a scheduler (which decides what tasks to run and when) and a rebuilder (which decides when to build/rebuild artifacts).
The paper specifically discusses Bazel as one example, where it uses a “restarting” scheduler (if a task is stuck because a dependency isn’t ready, kill the task and restart it from scratch later) and a “constructive trace” rebuilder (storing the results of prior steps, not just using a hash or setting a dirty bit).
From the original email announcing the intention to upstream, by Philip Herron (emphasis mine):
[…] my current project plan brings us to November 2022
where we (unexpected events permitting) should be able to support
valid Rust code targeting Rustc version ~1.40 and reuse libcore,
liballoc and libstd. This date does not account for the borrow checker
feature and the proc macro crate, which we have a plan to implement,
but this will be a further six-month project.
It’s important to note that GCC Rust will be initially marked as “beta”.
As I am aware, it will influence one particular thing: platform support. A significant amount of embedded toolchains are still GCC based rather than LLVM, and a number of niche architectures have better supported GCC backend (or backends where LLVM may not have any). That said, for esoteric architectures Rust itself may not be suitable due to guarantees the language requires (for example, a requirement that signed integer types in release builds either panic or wrap around on overflow more or less requires a two’s complement implementation for performant code).
People say that Copilot is the same as a human learning from reading GPL code or whatever, but humans don’t produce long verbatim transcriptions of the fast inverse square root. It’s not the same. Microsoft should not have included GPL code in Copilot. It’s a PITA that they did because it just muddies up all the water and makes what should be a useful tool into a potential legal time bomb. They played fast and loose and now everyone else is in legal limbo until it gets resolved in the courts.
GPL isn’t special here. Unless the code is CC0 or WTFPL or whatever, reproduction of substantial parts with no credit is a violation. Basically all OSS code is under a license that has restrictions on reuse.
Violating the MIT license strikes me as less worrying because you’re only distributing the license to make sure that the lack of warranty is known, but yes, that’s also a concern.
Maybe they could make a reverse Copilot that looks at a snippet, figures out where it is substantially from, and adds a license comment. 🙃
Speaking as someone who has released a load of MIT-licensed code, there are two things that I want to get out of releasing it:
If it’s useful to other folks, it’s easy for them to contribute things back (saving me work). Attribution matters here so that they know where to send bug fixes.
It helps build my reputation, which makes it easier for me to get people to pay me for other work (including adding features to the MIT-licensed projects).
It’s not clear that the warranty disclaimer is even necessary anymore. It was based on a ‘80s view of copyright and contract law. These days, there’s very little expectation that software comes with an implied warranty (even less if you’re not paying anything for it) and so the disclaimer of warranty is probably (though not definitely) superfluous. If someone did sue you over bugs in code that you have away for free, it’s likely that the court would require you to give them a full refund and make them pay their own legal fees. The warranty disclaimer bit is a CYA in case this interpretation of the current legal climate is wrong (or in case it changes in the future). In contrast, the attribution is core to the reasons that I publish the code in the first place.
I’m proud to have created the Rust Language Server (RLS) with @nick_r_cameron
and equally proud today to see it deprecated and replaced by Rust Analyzer. It
served its role well, helping to kickstart IDE support for @rustlang.
gibo, mentioned in the post, is pretty cool. It dumps GitHub’s recommended .gitignore file contents out, so you can kickstart a .gitignore for a new project.
It’s one to the shell utilities that I install when I set up a new development box. For this post I went with the curl since it’s Python specific and an easy copy/paste. The reality is that I end up typing gibo dump Python more often that ctrl+r /gitignore.
This was my first time doing a public talk. I tried to summarize a pretty big topic in a short time, I hope the details didn’t get too lost. Any feedback would be much appreciated!
Thanks for the talk, it was interesting! Here’s some feedback about how the presentation felt for me:
do, or LINQ or Elixir’swithexpression didn’t come into it, cos they seem like big usability benefits.In the Q&A you talked about “subtype inference”, which sounds a lot like the structural typing in Typescript and similar to the structural typing in Go and OCaml.
Thanks for the feed back! Those are very valid points.
I didn’t talk that much about the usability of algebraic effects, because right now, there isn’t much. Of course the entire comparison is unfair since Monads are well explored while Algebraic Effects aren’t. And since I wanted to keep this somewhat short and understandable, a lot of cool things about Monads didn’t get mentioned.
Excellent talk, great introduction to monads & algebraic effects!
You touched on the fact that before monads, Haskell used streams for all IO. You mention that this API was quite weird/annoying, and other than your talk I’ve only seen some very high-level descriptions of this online (which I remember coming to roughly the same conclusion). Do you happen to know what actually made this API so weird/annoying?
In the current codebase I am working on, we use RxJS observables for all app state & communication between functions/modules (in a similar pattern to Cycle.js, if you are familiar with that). In this world you have functions that transform source streams into sink streams, and I think this is actually a really nice pattern in my opinion. Dataflow is 100% explicit, ie. you can tell what external resources a function can access just by looking at which streams are passed in as parameters.
Having worked with Haskell previously, I actually prefer this style of IO: it’s much easier to combine multiple input streams from different sources (ie. “read effects”) using stream operators than composing multiple monads (you touched on this with monad transformers).
On your blog, I saw that you mention Pointless, which takes a similar approach: its equivalent of
main(output) is a lazy list of commands. Elm also has a similar idea for IO, with ports.So yeah, wondering if you had any insight on why Haskell moved away from this - are we just re-inventing the wheel badly?
I don’t, or definitely not more than you already have. I’ve never tried this myself, and mostly just repeated what I’ve read in the HOPL paper on Haskell, in section 7: https://dl.acm.org/doi/10.1145/1238844.1238856 Also relevant is the video from the HOPL conference starting at minute 16 or so. There it’s called a “prolonged embarrassment”.
I also read through the relevant sections in the Haskell Report 1.0 and 1.3. They are surprisingly short! But still, the monad version is simpler and shorter. But I do feel like stream based effects might not have gotten a fair try in early Haskell. The “continuation-based IO” feels like the wrong abstraction. While I was sometimes annoyed by Elm’s ports, they are absolutely practical and very far from a “prolonged embarrassment”. It also feels like these effects as streams fit UI very well. Especially web UIs, where you (mostly) don’t have parallelism.
For anyone wanting to read the paper without an ACM Digital Library subscription, here’s a freely available copy hosted by Microsoft’s research blog.
And the evolution of the the Haskell I/O system is summarized with code examples on p. 24 of the PDF.
Agreed on that repeating the meme is just being too lazy to call out which part specifically is broken.
That said, by far the biggest problem I have with Python dependencies is failures during installation of packages that compile C/C++ code that break on wrong headers, libraries, compiler versions etc. Most often with numpy, scipy and similar.
Now, is there hope the situation improves without migrating off of
setup.pyscripts? I doubt it. Everything needs to migrate to something declarative/non-Turing-complete so that those edge cases are handled in the tool itself, once and for all packages. I’m not sure ifsetup.cfg/pyproject.tomlcover all the use cases to become a completesetup.pyreplacement. Probably not (yet).Yes, that’s the real issue. Django web devs can happily play with pure Python packages all day long and wonder why others are complaining until they try to build a package with C code from source.
Numpy, for example, goes full clown world – last time I checked you needed a Fortran compiler to build it.
The highest-performance mathematical libraries are written in Fortran, so numpy using it is pretty much a requirement for their use case.
God have mercy on your soul if you try to pip install a package with C dependencies on a more niche Linux system (maybe this only applies if you’re not using precompiled binaries?).
Niche Linux system I’ve had trouble compiling C dependencies on: Termux on Android.
But the picture on Windows is even worse. If a C dependency doesn’t have a binary wheel, you may as well give up.
I don’t really get this. I can
pip install numpywithout a Fortran compiler. So can you! It has pre-built binaries, on the Python Package Index, for a bunch of popular operating systems and architectures. I can’t remember the last time I needed to actually build a package from source just to install and use it.This is not quite inaccurate. Try to install it for example in alpine linux. Or try to install a very new version. Or use a less common feature not included in the default pre-built packages
’‘pip install’’ does rely on compilers for countless packages.
The packaging ecosystem supports (as of PEP 656) binary packages compiled for distributions using
musl. So if you find a package that requires you to install a bunch of compilers on Alpine, it is explicitly not the fault of Python’s packaging tooling – it’s the fault of a package author who didn’t provide a built binary package formusl.As for “very new version”,
numpyalready appears to have built binary packages available on PyPI for the as-yet-unreleased Python 3.11.And if you’re going to customize the build, well, yeah, you need to be able to do the build. I’m not sure how Python is supposed to solve that for you.
No. You already explained in your second paragraph what the problem is. All this talk about poetry or praising package managers of other languages as if the better experience from those wouldn’t heavily rely on the fact that those who use them, do so in a very limited scope.
Remove all the packages that rely on the binary C libraries that do the heavy lifting, et voila, you are at the level of smoothness as other package managers. I would even argue that python virtual environments are unmatched in other languages, if anything.
Which other language provides a high level interface to cuda with a easier set up? Which other language provides functionality equivalent numpy with a smoother setup?
C libraries are installed in different ways in different systems. Pip will try to compile C and this will horribly degrade in the absence of the exact ABI on some dependency the package expects.
I am not really sure this has a solution. Support for different operative systems is important.
You’ve made a couple of points which I don’t necessarily agree with but let me just focus on the original one in this thread.
As @ubernostrum has pointed out in a different thread, it’s best if a package comes with a precompiled binary wheel (and statically linked at that). Installation is great then and it’s probably the solution. That said, I can’t install e.g. PyQt5 (i.e. not a unpopular package) on a M1 Mac at the time of writing this.
curl https://pypi.org/pypi/PyQt5/jsontells me there’s no apposite binary package for it.The next best thing IMO is to vendor C/C++ library source code (either through a git submodule or even try to download the sources on the fly maybe) and attempt compiling it so that it doesn’t depend on any system-installed headers and libraries. This presupposes a presence of a working C/C++ compiler on the system which I think is acceptable.
What’s fraught with peril though, is to assume that the relevant library (along with its headers) is preinstalled on the system and attempt linking with it. This is just too complicated of a problem to be able to handle all the cases correctly. (1) It relies on the presence of some external means of installing libraries, so it conceptually takes you outside of the Python package manager into
apt/brew/etc territory. (2) Your Python project can break when you upgrade the library if you dynamically linked with it, or it can break silently due to an incompatibility if you’re linked statically with it (equally bad). (3) The Python package has no control over the exact version of the system dependency so caveats abound. (4) Your Python project can break if you uninstall the system library, since there’s nothing preventing you from doing that.If I could change one thing about the the Python ecosystem, it would be to push for the first two solutions to be preferred by package authors.
Yeah, at some level, Python’s deepest packaging problem is that Python performance stinks, so you need to use C-wrappers for math and whatnot, but once you do you run into the fact that however bad Python packaging is, at least it exists, which is more than C/C++ can say.
OTOH, Zig is just one guy, and I haven’t used it personally, but it seems like it can handle a lot of C cross compilation stuff? So ideally, the Python core team should just say “C/C++ is part of our domain, so we’re going to include the equivalent of zig build with CPython to making packaging work.”
Just FYI, Zig has a language creator but it is not “just one guy” any longer. It is being developed with the help of the ZSF, https://ziglang.org/zsf/, which is now paying for core contributions beyond the language creator. There is also a small group of core committers contributing to the language and fixing bugs. It is being very actively developed and their next release will be the “self hosted compiler” (the Zig compiler will itself be written in Zig and built with Zig), which will likely also make it easier for new contributors to join the project. There are also some core contributors beyond the language creator who have started to specialize on Zig’s use cases as a x-platform C compiler, for example by focusing on areas like libc improvements (shorthand: zlibc) and on the various target platforms beyond Linux, like macOS, Windows, WebAsm, embedded, etc. It’s likely true today that Zig is the world’s best x-platform C build toolchain, and that will likely only get better and better over time due to the vitality, pace, and mission of the open source project (especially compared to the alternatives).
Your idea about how Zig could help the Python C source build process is a good one. There is a nice post here about maintaining C code and C cross-compilation issues with Zig:
https://kristoff.it/blog/maintain-it-with-zig/
Unfortunately some Python code relies on C++, Fortran, and other exotic languages to build sources, since the Python C extension module interface is so open and so many other languages can expose something that speaks the C ABI. But you’re right that C modules written in C are most often the culprit.
The creator of the language also talked about some of the technical details behind
zig cchere:https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html
I think that’s a fantastic way to improve the Python packaging status quo 1, 2! Zig is a game changer when it comes to portability.
This is just wrong, and it’s frustrating to hear it repeated so often.
unsafepermits calling otherunsafecode, dereferencing raw pointers, reinterpretingunions, and implementingunsafe traits likeSendandSync. It does not “turn off” the type system, borrow checker, or anything else.Was also quite unhappy seeing this incorrect understanding of
unsaferepeated.My favorite explanation is Steve Klabnik’s “You can’t ‘turn off the borrow checker’ in Rust”.
Though you can do:
Which makes Rust forget that
myrefis borrowingself. Not turning off borrow checker, but at least limiting how much it “knows”.No it’s not. It’s just a
Resulttype. But I understand that it may seem that way if Rust is the first post-1995 language you’ve ever used.While I understand this isn’t the most technically advanced article out there, all I’m trying to do is provide the same learning moments I had to other people in similar positions to me.
The best teachers are often those who just learned something, and remember what it was like not to understand it.
Keep at it.
Just to add to what @Loup-Vaillant said above, one of the things Rust is sorely lacking is alternative learning paths, driven by the experience of people from various backgrounds. Rust is so large and intimidating that very few people choose to write about it in a manner that’s not just rehashing official documentation with some advocacy sprinkled on top. Your post involves a bunch of cool idioms related to a very practical problem. It’s a very good post that fills a very relevant niche, not just with PLT in general, but also with Rust in particular!
Don’t be a dick.
All
Resulttypes are cool, including Rust’s.If Rust makes it possible or attractive for people to use 21st century lang features, I’d call that a success for Rust and for programming in general.
This particular feature is squarely XX century though. A quote from first rust presentation is apt:
That’s quite surprising and important to tactfully highlight !
I object-level agree that the Result type is cool - I thought precisely the same thing a decade ago when I learned about Algebraic Data Types and specifically the
MaybeandEithertypes in Haskell. That isn’t a joke, I really was blown away by these programming constructs in Haskell when I first learned about them there, and I was surprised that no programming language I had ever previously used had support for what were obviously such good ideas. I still think that one of the most exciting things about Rust is that it popularized ADTs in a mainstream general-purpose programming language (not to say that Haskell is obscure, exactly, but Rust is probably more widely used than Haskell at this point).So I’m inclined to agree that Rust’s
Resulttype shouldn’t be conceptualized or taught to beginners as some special Rust thing, but rather as a (genuinely quite generally useful) application of a well-established programming language construct that Rust happens to make use of. And it’s definitely worth questioning why it took so long for the ability to create an Maybe-like ADT to become a mainstream programming tool, and to wonder about what other broadly useful programming conceptual tools exist, but just aren’t available in the mainstream programming languages that a lot of the world’s software gets written in.Either (and Haskell data types in general) are just tagged unions, which plenty of languages have. You could argue that inheritance based polymorphism is the OO language equivalent.
Obviously every older imperative language has Maybe as well in the form of pointers - the problem of course is that all pointers are always maybe and there’s no mechanism to ensure you check before using :)
I’ve recently started using a library for C# called OneOf, which makes it easy to implement Result and Option types, and it’s really cleaned up my error handling, especially solving the “never return null when the caller expects an object” trap that C# and Java dump you in by default. There’s little/no syntactic sugar in the language to help with it, but as a lisper, it doesn’t really bother me that match is a method rather than a keyword.
Sure, but calling basic features “cool” is juvenile.
Why? Do you never go “huh, that’s cool” to anything anymore?
To be honest, I never understood what “cool” means, except as something that kids say, so no, I never used the term, as a kid of otherwise. But when something is described as cool, it’s a very good litmus test for me signalling it’s something I don’t need to pay any attention to.
That’s disappointing as it’s not the signal you seem to think it is, so you might well be missing things you would have otherwise found interesting.
You are so wise and smart and above it all.
Think twice before posting again. Is your contribution useful and kind? If not, refrain from expressing yourself. No one is getting paid here. You get what you give,
True, necessary, and kind, yeah? Worth trying to hit at least two of those three.
I’d be curious if the team thinks this gives a positive or negative experience to users.
One of the few unquestionably good experiences I’ve had working with a large Go project is that
gofmtprecludes all arguments about style. rustfmt has knobs for days, and I wonder how much turning they get.The bigger difference between rustfmt and gofmt are not the knobs, but the fact that gofmt generally leaves new line placement to the programmer, while rustfmt completely disregards existing layout of code.
That’s one of the most frustrating experiences writing go code for me. I’m used to automatic line wrapping.
OTOH to me rustfmt’s “canonicalization” is a destructive force. There are many issues with it. One example: I strive to have semantically meaningful split of code into happy paths and error handling:
To me this gives clarity which code is part of the core algorithm doing the work, and which part is the necessary noise handling failures.
But with rustfmt this is pointlessly blended together in to a spaghetti with weirdo layout like this:
or turned into a vertical sprawl without any logical grouping:
rustfmt doesn’t understand the code, only tokens. So it destroys human input in the name of primitive heuristics devoid of common sense.
I want to emphasise that rustfmt has a major design flaw, and it’s not merely “I don’t like tabs/spaces” nonsense. The bikeshed discussions about formatting have burned people so badly, that it has become a taboo to speak about code formatting at all, and we can’t even criticise formatting tools when they do a poor job.
As a user, I like that rustfmt has some knobs, it lets me adjust the coding style to use within my own project, so it’s internally consistent.
What matters to me, as a developer, is that the coding style within a project is consistent. I don’t care about different projects using different styles, as long as those are also internally consistent, and have a way to reformat to that style, it’s all good. I care little about having One True Style. I care about having a consistent style within a project. Rustfmt lets me have the consistent internal style, without forcing the exact same style on everyone.
It’s worth saying that Rust is targeting C and C++ programmers more than Go is, and at least in my experience it’s those programmers who have the most idiosyncratic and strongest opinions of what constitutes “good style.” “My way or the highway” may not work well with many of them.
Go was originally intended as an alternative to C & C++, fwiw.
Happy that this finally shipped. I almost unsubscribed from the feature request discussion for this, even though I wanted to keep up with the status. The team ended up delayed shipping this feature and people are apparently unable to refrain from asking for status updates.
Yeah the constant “hey when is this happening” questions were unhelpful. The team did give periodic updates on the delays, along with confirmation that it was being actively worked on, and estimates of when it may be done, which is more than a lot of organizations give for new features.
I agree, the team did a great job on their part. :)
This a cool example of the severe limitations of microbenchmarking. In microbenchmarks, the performance impact of the extra function call is present, but it completely disappears in larger benchmarks due to JIT optimization. This is also good evidence that benchmarking should be done in the context of real applications and with a focus on what’s actually being observed as slow on real workloads.
The description of how to make a trampoline in Rust to translate from the Go calling convention to the Rust calling convention and back is great! Also a cool example of why the
asmmacro and naked function attribute are so useful!Thank you! Appreciate the feedback
btw mirrord looks super useful. TIL.
Doing the main work for my Big Data final project, which for me means some basic analysis of Google’s Open Source Insights dataset to understand the relationships between project popularity and vulnerability rates and severity. Should be fun!
Yes, finally! Definitely love the crate and honestly surprised that the name was available still.
An amazing history of an amazing set of tools, with a finishing line where I realize I’ve been mispronouncing it for years. Cheers to Valgrind (pronounced correctly now), and cheers to Julian, Nick, and everyone else involved in making it all work!
This is sneakily an introduction to the value of recursion schemes, which are fantastic.
For the unfamiliar, recursion schemes are a way to describe patterns of recursion separately from the operation to do at each recursive step. In the final version of this code,
generateis the recursion scheme, andgenerate_from_boxedis the operation to do at each step.I think part of why recursion schemes have had difficulty breaking through is that they’re named terribly. “Catamorphism” and “anamorphism” are the two basic ones, and the names only get worse from there. Seeing something like this makes me hopeful about the possibility of recursion schemes gaining wider adoption by abandoning the old way of teaching them!
I think terrible naming and other communication issues hold back just about everything in the PLT world. Cue the old joke about “Monad is just a monoid in the category of endofunctors”. I certainly why some people perceive PLT folks as having a potent “ivory tower” culture.
Yeah, this is part of why I’m glad Rust didn’t start out with support for higher kinded types. If it had them from day one, people likely would have directly imported
Monad,Applicative,Functor, et al from the start, tying Rust to these terms which (imo) ought to be left behind.I agree with this,
Monadetc are heavily loaded terms at this point. I think a more effective way to motivate people is to show the benefits of these abstractions well before delving into the theory.If we leave these terms behind, then who will write the dozens of Rust-oriented (as opposed to Haskell-oriented) articles trying (and usually failing) to explain what a Monad is? /s
More seriously, naming is hard. And I’m sure there would be a lot of pushback from not naming things as they are in the most popular functional languages.
What’s a CS degree like these days, anyway? Back when I was in school decades ago, I don’t recall us covering much if any theory of programming. We just jumped right into structured programming, data structures and all that.
On the contrary, I think for the right language will bring the functor/applicative/monad abstractions to the mainstream, probably not under those names, however. Angry people in comment sections/social media are not the whole world, thankfully.
This sentiment is maybe eight decades too late. The words “functor” and “monad” come from philosophy, borrowed from Carnap and Leibniz respectively. The word “category” is also borrowed, from Plato and Aristotle. You can suggest new words if you like, but perhaps it’s worth considering why the original category theorists borrowed existing words instead of making up new ones.
The idea of “the right language” suggests that you’re thinking exclusively of endofunctors, internal monads, and other internal structures within a single programming language. However, it’s important to recall that every language has its own category of types (via native type theory), and so we have functors from one language to another. This abstraction is not language-specific and has been studied for half a century under the name “functor”. Here we could have another nomenclature bikeshed; surely we should call these “compilers” or “translators” instead.
How would you teach monads, and what would you call them instead of “monads”?
Depending on the situation, I would call them
flat_mapor similar. I would teach them as something like “say you have a tree of computations: if values in the tree can generate new nodes in the tree, then that’s a special case that you need to think about”.How would you express the algebraic laws? Also, how would your approach differ from the monoid-of-endofunctors approach?
I think that your comment, as is, is baseless: it makes a claim, which is fine, but then it tries to support it by example which is not convincing. You take a sentence of category theory as an example of communication issue in Programming Language Theory. The sentence is appropriate for audiences in some context (… people interested in category theory) and not in others; it is not in itself a communication issue, it all depends on the context in which it is used.
There are many interesting things to be said about the interplay between various communities (academic and non-academic ones in particular) on programming language design and implementation. But that requires nuance and going above tropes such as the ivory tower. Repeating it without providing any such nuance does not improve the discussion in my opinion, it is impoverishing.
P.S.: this being said, I totally agree that zylomorphisms, zygomorphisms, patamorphisms etc. are impossible names, they sound satirical to me and should probably be avoided. (Histomorphism is somehow slightly better)
Yes, that’s the point of the joke (and I did explicitly refer to it as a joke). People in the PLT world try to explain concepts to outsiders in terms that are only meaningful to insiders. It’s a widely-known and relatable joke among programmers, suggesting that there may be a communication problem. You’re free to disagree, of course–I certainly can’t prove it one way or the other, nor am I interested in trying.
I was noting communication problems impede PLT ideas from achieving adoption in industry, I wasn’t attempting to examine interplay between communities. I may still have failed at my goal, but I don’t see the point in telling someone they failed at something they weren’t attempting in the first place. 🙃 🙂
So a large group (programmer) widely spread a joke meant to criticize a subgroup (programming language theory people), and from this you conclude that the subgroup has a communication problem. Many other conclusions would seem reasonable, including that members of the larger group are prejudiced against the subgroup or dismissive. Explaining this joke away as a “communication problem” from the subgroup seems to ascribe fault in a very one-sided way in this social interaction, when I think again that the situation requires more nuance. (For example: some PLT people do come off as elitist or condescending based on their attitude, and in some cases their choice of words, but there is also a complex relation between programmers and formal/theoretical programming education, combined with an unhealthy tendency to expect wonders from some ideas (including category theory) with hype-disappointment cycles, etc.).
One community: PLT people communicating on their work. Another community: programmers in the industry or free software projects.
Obviously I didn’t assign fault for the communication breakdown. Happy trails! ✌️
Thank you! That was my goal writing this, to motivate people to learn recursion schemes in a way that felt like it naturally falls out of building a cache-local evaluator for recursive expressions. I was delighted to find out that it does naturally fall out of building a recursive evaluator in another idiom, which in retrospect makes sense - of course there would be deep connections between different ways of expressing recursion.
My knowledge of Greek teaches me that “anamorphism” and “catamorphism” mean “up shaping” and “down shaping”. I get that there are a lot of cases of simple Anglo-Saxon terms being turned into jargon made up from Latin/Greek stems, but “up/down shaping” (or up/down folding) seems like it’s just as good as a jargon term. Like in literature, “katabasis” is used to distinguish a character going down into the metaphorical netherworld vs. just going down the stairs, but I think for category theory, “down folding” has enough context that it doesn’t need a Greek term.
Recursion schemes already broke through and are mainstream. Katamorphisms are available in many popular programming languages.
I’m not sure I understand your complaint. Is the issue that the “old way of teaching them” is not clear, or is the issue that “they’re named terribly”?
You’re right that many languages offer fold, and fewer also offer unfold. My issue is that the space of recursion schemes is much richer than these, but the obtuseness of the existing terminology and lack of good educational material which sidesteps it has limited the spread of recursion schemes as a concept beyond the fold/unfold functions.
I see. For what it’s worth, I think that the issue is that the typical popular programming language was not designed, but copied from previous languages. When a language is designed from scratch, there is an opportunity to improve, but most languages do not get that opportunity.
For example, one of the recursion-schemes authors worked on C♯, but rather than a deeply-integrated object protocol which adds folds and unfolds to each collection, they developed LINQ. (They did also add folds to every collection with an extension method, based on the ability to enumerate it; this is similar to how Haskell’s prelude changed to allow all collections to be Foldable.)
IME most languages that offer fold/unfold only offer it over sequences of values, and not structured data like trees or user-defined types.
A real problem with string_view is that by design it isn’t memory safe: you can return one from a function, store one in a strict, or whatever and it will silently carry a pointer to its backing store, whether or not the backing store is kept alive.
Personally I think it’s a foot gun of an API given how easy it is to use it in a way that results in UAF bugs.
This feels slightly unfair to call out
string_view, iterators have all the same problems, do you consider them a foot gun of an api too? It is a foot gun, but it’s not any more of a foot gun thanchar *and maybe only slightly more of a foot gun thanstring&.And it at least has the advantage of consistently being a non-owning reference to the backing store, something that could go either way with
char *.In the context of C and C++,
string_viewis an improvement over alternatives. However, Rust has shown the value of statically guaranteeing no dangling pointers. Rust’s&str(the equivalent of C++‘sstring_view) can’t point to a string which is no longer valid to reference. That’s pretty great!Sure. And if the original post had said something like “it’s too bad C++ doesn’t allow string_view to be memory safe by design.” I probably wouldn’t have made an objection. This is a “real problem” with vast swaths of C++ apis though, it is not notable or exceptional that string_view suffers from it.
Sure, in the context of C++, it’s not notable that
string_viewhas this problem, but in the broader context of this set of competing languages (C, C++, Rust), it’s worth noting when a new API is offering less than state of the art guarantees.But by focusing on the api the implication is the specific api is flawed as opposed to the language. No new C++ api will ever be “state of the art” without a language change. We’re not talking about adding C++’s string_view to Rust where it would make sense to criticize the api for not being memory safe, where the option to do better exists.
I’m curious about how this works. If in Rust, I have a reference counted object A, that points to a reference-counted object B, that has a string field, and I use
&strto take a reference to a substring of that string field. I then pass A and the substring to a function, which follows the reference from A to B and truncates the string, how does Rust enforce the bounds checks? If I completely invalidate the range that the substring covers, what happens when I try to use it?I believe the answer is that the Rust borrow checker would prevent objects reachable from A from being mutated during the function that has the substring reference and so I can’t express the above idiom at all. In C++, there is no such restriction and you get all of the power, flexibility, and the footguns that accompany this.
I think (and could local Rust experts please correct me if I am wrong; @matklad, @kornel) Rust will not let you both have a mut reference from A to B and an &str to substring in B since that would mean you have two references and one of them is mut. You could have a non-mut reference from A to B and an &str substring to B but that means you wouldn’t be able to modify B in your function. But I could be complete wrong.
Yeah, this is mostly correct, with the asterisk that a) what Rust really tracks is aliasing, rather than mutability, and it is possible to use interior mutability to mutate aliased data and b) you can use
unsafeand raw pointers to gain all power, flexibility, and even more footguns.Couple of examples here:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=88c9e646c242c510d4fc8a52871cc323
I consider atring_view to be a new API introduced well into the period of knowing that dangling pointers are bad. I couple that with it behaving as a value object rather than iterators that have pointer semantics in my belief that string_view is bad.
Iterators date to the earliest days of c++ and have a well understood pile of issues that make them not memory safe (the for(:) using iterators removes a lot of safety checks and introduce vulnerabilities from previously safe code with indices)
Modern C++ APIs are not designed to eliminate memory safety bugs, they are designed to permit coding styles that eliminate pointer safety bugs. Combined with the lifetime annotation on accessors, a compiler can warn if you have locally created a string view that outlives the object from which it was created. A coding convention that says not to capture string views (pass them down the stack only) then gives you most of the lifetime safety that you need.
The corner case is if the object that the string view references is reachable from another object and the called code mutates it in such a way that it invalidates the string view. For string views derived from something like
std::string, this could be avoided by having the string view reference the string object and its bounds but that adds performance overhead and would reduce adoption and it cannot work with string views derived from C strings: One of the big benefits that you get from string views is the ability to provide safe implementations of APIs that are exposed to C and take C string arguments and can also be used from C++ with string objects.But new APIs should be designed such that they make it very easy - to the extent I’ve seen multiple blogposts and mailing list comments on the memory unsafely of string_view when used in a way that looks reasonable. If a new object is added that is trivially unsafe if moved, copied, or otherwise stored is introduced, then that object should not support copy or move.
One of the more common errors in llvm is storing the semantically equivalent StringRef, which often works fine due to other objects keeping buffers live long enough. Except when they don’t, and then you get crashes or worse not crashes but whatever happens to be present at the time.
It is not reasonable to add new APIs to C++ where code that looks correct is not memory safe.
If you remove the copy constructor in
string_viewthen you eliminate its only valid use: passing down value type down the stack to other functions for temporary delegation. The problem is that C++ doesn’t differentiate between copy and capture, so there is no way of implementing a thing that can be passed down but cannot be captured. You could force it to be passed by reference, but then you’re adding an extra indirection for a two-word object and that will hurt performance.The main reason that string_view exists is to have an explicitly non-owning type that replaces passing a
char*and asize_tin a way that makes it easy for one of them to be lost (for other code to assume that thechar*is null terminated, for example).As you say, llvm’s
StringRefis analogous and any code that captures aStringRefis a bad code smell and gets caught in auditing. Prior toStringRef, code either defensively allocatedstd::strings all over the place and hurt performance or decided that performance was important and capturedconst char*s.StringRefhas been a huge improvement over the prior state.If you have a proposal for a performant, memory-safe, non-owning string reference type that could be implemented in C++ (even with some modest language additions, but without completely redesigning the language) then I’d be very interested to hear it because it’s something that I’d happily adopt in a bunch of codebases.
No, if you remove copy you force pass by reference.
That has the benefit of meaning you can’t make a string_view outlast its lexical context, and you restrict it to what it ostensibly is: an abstraction API over the many variations on what a string is. In exchange for having to put an ampersand in you remove - or at least significantly reduce the footgun-ness of the API and potentially improve perf+code size as you can pass a pointer/reference in a single register :D
But you also allow capture by reference for lambdas, for example, so you’re still not memory safe.
True, but that just moves the problem. You can trivially make the reference outlive its lexical context, for example by lambda capture, by using
std::forward_as_tupleto capture a parameter pack, and so on. Worse, the things that capture now look a lot more like things that are safe to capture and so it’s harder to spot in code review.You also introduce a load of more subtle lifetime issues. By passing
std::string_viewby value, you can construct them in arguments from things likeconst char *orstd::stringreferences. This makes it very easy to gradually transition an API from one that takesconst std::string&andconst char *to something uniform. Ifstd::string_viewhad to be passed by reference then creating a temporarystd::string_viewand passing it would be very dangerous because the lifetime of the temporary could very easily be shorter than the use, even in the common case where the underlying string had a longer lifetime. Auditing code for this would be much harder than spotting misuse ofstd::string_view.If you want a safe string view, then it needs to be integrated with string memory management and prevent any mutation of a string that has outstanding string views. That would be impossible to integrate with APIs that exposed
const char*parameters and so would not have the incremental migration benefits of the current design.So is your argument string_view just shouldn’t exist? “by design” implies to me you think there is an alternative design for string_view that would be memory safe.
string_view is tantamount to a pair of iterators, I’m not sure how that makes it a value object and iterators have pointer semantics.
My argument is that the existent of bad APIs in the past should not be used as a justification to add new unsafe APIs.
The fact that the implementation is/may be two pointers is irrelevant, it could also be a copy of the data, it could be a pointer and a length, it could be a unordered_map from size_t char, the implementation is entirely moot.
The string_view API behaves like a value type - it looks (in use) to be a memory safe value type like std::string. The iterator API is via the standard pointer operators *, ->, etc. All things that display very clearly that they are semantically only pointers to data, nothing else.
What matters is how a type behaves, not your thoughts about how it should be implemented.
Your argument is “C++ is filled with things that are unnecessarily unsafe because of backwards compatibility, therefore it is ok to add new unsafe things”, which is not compelling to me.
My argument is it is impossible to add a memory safe string_view to C++.
These things aren’t “unnecessarily” unsafe, they’re intrinsically unsafe.
The argument from a memory safety standpoint is “just don’t use C++”, not “don’t add things to C++”. Adding string_view does not make C++ less safe.
string_view can’t be a copy of the data because then it ceases to be a view, you would just use string then.
You could make string_view not movable or copyable, and the problem is solved.
It is? How do you implement
string_view::substr? I suppose it could besubstr(pos, n, LambdaTakingConstRefStrinView f) -> decltype(f(declval<const string_view&>()))And
is still obviously a UAF even with a non-copyable non-movable string_view.
Though I suppose to solve that problem we could again use the same continuation trick, and not make the
char*ctor public:Yes you can explicitly free the backing store but there is a world of difference between safety in the face of explicitly releasing backing buffer - what you’re suggesting is not that far from
shared_ptr<std::yolo> ptr = …; ptr->~yolo();The specific issue I have with string_view is the ease with which it makes UAFs possible, without apparently doing anything sus.
I think I said elsewhere, but LLVM has llvm::StringRef which has the same API semantics, and it is a frequent source of memory bugs - only sometimes caught during review. It has the same issues: it often appears to work due to incidental lifetime of the backing store, or the general “releasing memory doesn’t magically make the memory cease to exist” behavior that makes manual memory management so fun :)
Sigh, why does lobsters occasionally hold off multiple days before it says “there are replies to your compelling commentary!” and make me seem obsessive.
This criticism totally depends on what you use instead! If the alternative is a pointer or reference, they obviously have exactly the same problem – they can outlive the memory:
I’ve heard this criticism before, but I don’t think it’s relevant with the right mindset: It isn’t a string replacement! As a slice data type, it’s just a fancy reference. It exists to replace references and pointers, not owned strings.
As the article mentions, a good use is function arguments. The reason is that the caller only needs to make the memory outlive the call. This may seem like a problem if the function needs to store the string, but that’s not a (valid) use case, as then, it needs an owned string, and should therefore take an owned string.
It is as a unifying replacement for other pointer-ish interfaces that it really shines:
C++14:
C++17:
For any Rustaceans, I built
woahfor exactly the need this article lays out. It provides a typewoah::Result<T, L, F>, whereLis a “local error” you can handle, andFis a fatal error you can’t.woah’sResulttype propagates fatal errors up with the?operator, giving you astd::result::Result<T, L>to handle local errors.It’s pre-1.0 until the
Trytrait is stabilized, but can be used on stable if you’re willing to forgo the?operator and do the propagation manually.Managing multiple different levels of errors is still difficult for me in Rust. I’ll take a look at the crate but I’m wondering if you can explain what the advantage is of using that over
Result<T, LayerError>whereThe difference is in having separation between the “local errors” and “fatal errors” and how that interacts with the
?operator.woah::Resultis equivalent toResult<Result<T, L>, F>in terms of how it works with the?operator. Take the following example, copied from the project’sREADME.md:The key line is the
match do_http_request()?. The?operator will pass anyFatalErrorup to the caller ofget_data, and return astd::result::Result<String, LocalError>which is whatget_datais matching on. In the error case on that match, we have aRequestTimedOut, so we can retry.I highly recommend anyone interested in this article also read the paper “Build Systems a la Carte”. You can skip/ignore the Haskell bits, and focus on the description of build systems as a combination of a scheduler (which decides what tasks to run and when) and a rebuilder (which decides when to build/rebuild artifacts).
The paper specifically discusses Bazel as one example, where it uses a “restarting” scheduler (if a task is stuck because a dependency isn’t ready, kill the task and restart it from scratch later) and a “constructive trace” rebuilder (storing the results of prior steps, not just using a hash or setting a dirty bit).
Extended version of the paper: https://www.cambridge.org/core/journals/journal-of-functional-programming/article/build-systems-a-la-carte-theory-and-practice/097CE52C750E69BD16B78C318754C7A4
Ah, I didn’t realize there was an extended version! Thanks for sharing :D
Is this the same project as this? https://rust-gcc.github.io/
According to this website:
I’m not sure if this frontend will influence anything without the borrow checker?
From the original email announcing the intention to upstream, by Philip Herron (emphasis mine):
It’s important to note that GCC Rust will be initially marked as “beta”.
https://github.com/Rust-GCC/gccrs/wiki/Frequently-Asked-Questions#mitigation-for-borrow-checking
They have a plan around it though:
(Polonius, for the uninitiated, is the next generation borrow checker for rustc that’s currently available as a library)
As I understand it (correct me if I’m wrong), Polonius is still both unfinished and very slow.
As I am aware, it will influence one particular thing: platform support. A significant amount of embedded toolchains are still GCC based rather than LLVM, and a number of niche architectures have better supported GCC backend (or backends where LLVM may not have any). That said, for esoteric architectures Rust itself may not be suitable due to guarantees the language requires (for example, a requirement that signed integer types in release builds either panic or wrap around on overflow more or less requires a two’s complement implementation for performant code).
This is no reason to build a new Rust frontend. GCC backend work is already underway and further along IIRC.
Clearly there is not consensus on that part, otherwise the work would not be receiving direct funding.
People say that Copilot is the same as a human learning from reading GPL code or whatever, but humans don’t produce long verbatim transcriptions of the fast inverse square root. It’s not the same. Microsoft should not have included GPL code in Copilot. It’s a PITA that they did because it just muddies up all the water and makes what should be a useful tool into a potential legal time bomb. They played fast and loose and now everyone else is in legal limbo until it gets resolved in the courts.
GPL isn’t special here. Unless the code is CC0 or WTFPL or whatever, reproduction of substantial parts with no credit is a violation. Basically all OSS code is under a license that has restrictions on reuse.
Or at the very least requires reproduction of the license by the user.
Violating the MIT license strikes me as less worrying because you’re only distributing the license to make sure that the lack of warranty is known, but yes, that’s also a concern.
Maybe they could make a reverse Copilot that looks at a snippet, figures out where it is substantially from, and adds a license comment. 🙃
Primarily to give credit / attribution, I would say
Speaking as someone who has released a load of MIT-licensed code, there are two things that I want to get out of releasing it:
It’s not clear that the warranty disclaimer is even necessary anymore. It was based on a ‘80s view of copyright and contract law. These days, there’s very little expectation that software comes with an implied warranty (even less if you’re not paying anything for it) and so the disclaimer of warranty is probably (though not definitely) superfluous. If someone did sue you over bugs in code that you have away for free, it’s likely that the court would require you to give them a full refund and make them pay their own legal fees. The warranty disclaimer bit is a CYA in case this interpretation of the current legal climate is wrong (or in case it changes in the future). In contrast, the attribution is core to the reasons that I publish the code in the first place.
I use MIT so that when I change jobs, I can still use all the code that I’m writing at my current job.
Jonathan Turner, one of the creators of RLS, had a nice tweet about this deprecation yesterday:
That is a wholesome response. Thanks Jonathan!
gibo, mentioned in the post, is pretty cool. It dumps GitHub’s recommended.gitignorefile contents out, so you can kickstart a.gitignorefor a new project.❤️ gibo.
It’s one to the shell utilities that I install when I set up a new development box. For this post I went with the
curlsince it’s Python specific and an easy copy/paste. The reality is that I end up typinggibo dump Pythonmore often thatctrl+r /gitignore.There’s also gitignore.io which allows combining and discovering different ignores
Saw it was written in Rust and wondered if it was already tracked on langs-in-rust, and it is (though I should update its star count)! https://github.com/alilleybrinker/langs-in-rust
Cool language; seems ambitious in its design!