Elixir is made up of very delightful layers of functionality. One time I wanted to submit a proposal to make the pipeline operator more general. I wanted to be able to pass in an underscore so you could specify where the piped argument would go. Something like 1 |> subtract(2, _). I was able to download the repo, find the code I needed to change and fix up the test cases in like an hour or two. It was very readable and easy to build. Would recommend checking it out.
There is built in way for that using Kernel.then/2 function in form 1 |> then(&subtract(2, &1)). I also have created magritte library that extend |> to support 1 |> subtract(2, ...) form.
Iâve had that same operator idea since forever. The restriction on where the argument goes with the pipe is relatively easy to work around but never feels general enough.
Are there any papers or posts that describe the current Pijul data model? Iâd be interested in reading about the underlying format. I know itâs undergone some revisions over the past few years.
What is the reason to not def send_emails(list_of_recipients, bccs=[]):?
From my Clojure backend I wonder what is sending the emails (and how you can possibly test that unless you are passing in something that will send the email), but perhaps there is something I miss from the Python perspective?
The answer is mutable default arguments; if youâre coming from Clojure, I can see why this wouldnât be an issue đ
When Python is defining this function, itâs making the default argument a single instance of an empty list, not âa fresh empty list created when invoking the function.â So if that function mutates the parameter in some way, that mutation will persist across calls, which leads to buggy behavior.
An example might be (with a whole module):
from sendgrid import actually_send_emails
from database_functions import get_alternates_for
def send_emails(list_of_recipients, bccs=[]):
# suppose this app has some settings where users can specify alternate
# addresses where they always want a copy sent. It's
# contrived but demonstrates the bug:
alternate_recipients = get_alternates_for(list_of_recipients)
bccs.extend(alternate_recipients) # this mutates bccs
actually_send_email(
to=list_of_recipients,
subject="Welcome to My app!",
bccs=bccs,
message="Thanks for joining!")
What happens is: every time you call it without specifying a second argument, the emails added from previous invocations will still be in the default bccs list.
The way to actually do a âdefault to empty listâ in Python is:
def send_emails(list_of_recipients, bccs=None):
if bccs is None:
bccs = []
# rest of the function
As for how you test it, itâs probably with mocking. Python has introspection/monkey patching abilities, so if you relied on another library to handle actually sending the email (the example above I pretended sendgrid had an SDK with a function called actually_send_email, in Python you would usually do something like
# a test module
from unittest.mock import patch
def test_send_emails():
with patch('my_module.sendgrid.actually_send_email') as mocked_email_send:
to = ['pablo@example.com']
bccs = ['pablo_alternate@example.com']
send_emails(to, bccs)
mocked_email_send.assert_called_once_with(
to=to,
subject="Welcome to My app!",
bccs=bccs,
message="Thanks for joining!")
This mucks with the module at runtime to convert sendgrid.actually_send_email to instead record its calls instead of what the function did originally.
The embedded computation looks like a trap which will make papers harder to reproduce. Is there support for directly embedding scripts from other languages?
There are built-in ways to use JSON data. I think for complex or expensive computations, users could leverage their preferred language, output JSON, and then pick it up as input in Typst.
I suspect that the moment Helix supports this format, I will start converting all my notes from Markdown.
I absolutely love the look of this, it seems to be everything Iâve dreamt of in a LaTeX replacement. The web UI is really nice too, although it seems to be struggling a bit under the load post launch announcement.
Are you writing lots of markdown in Helix? Whenever I try, I get annoyed by the lack of soft wrap. I suspect Iâd have the same problem writing LaTeX or Typst with Helix.
Helix has soft wrap in HEAD since #5893 was merged two weeks ago. If you run nightly, just add editor.soft-wrap.enable = true to your config. Else of course wait for the next release, though I donât know if there are plans yet for when it will drop.
Oh, and according to (nightly)hx --health, Helix doesnât support Typst yet. I guess it will be added quickly once there is a tree sitter for it.
Itâs weird to see a Rust UI post that doesnât at least mention Druid and related projects. Raph Levien and others have spent a long time thinking about why Rust GUI is âhardâ and the right data model to tackle the problem.
To me itâs not very clear what problem does Helix try to solve and what are the shortcomings of Vi/Vim/Neovim for solving that problem? I admit that I did not read very carefully the documentation of this project but I was expecting to get that information from the front page of the project.
code editor, rather than text editor: syntax tree is used for âtextâ editing and navigation (programming language aware âtext objectsâ), built-in LSP for semantics-aware features
more out-of-the-box: default config is reasonable, more features are built-in, editor tries to help the user by providing more introspective contextual help on how to use the thing.
more hackable, by virtue of being implemented in a collaboration-friendly language.
Sorry but Iâm not convinced that those are real improvements. Letâs take them one by one. First, I see no numbers to hold the claim that the new model is better than the old model. The thing that they give as an example in the link can be done in Vim with Visual Mode, more or less. So I would take it as a proposal for improvement, not as an actual improvement. Second, I would be interested in the complexity of implementing such a code editor. My guess is that it needs to ârecompileâ the code every time you make a change so that it can âreasonâ about your changes. This looks to me as an overkill for large files or large projects. More than that, old systems cannot afford this type of ârecompilationâ. Third, I donât understand your point here. Why more is better? Fourth, call me unagreeable, but I donât buy the Rust propaganda. Every project in whatever language it is implemented is âhackableâ and all programming languages are âcollaboration-friendlyâ. Iâm curious what you think about this.
I am not trying to convince you, you do sound like a person quite happy with vi, and thatâs perfectly fine! It feels like helix solves the problems you personally donât have. But I am pretty sure these are real problems for quite a few people (and I personally definitely acutely feel all of them).
Re editing model: I personally wouldnât rely on numbers to make a decision here. To me, itâs blindingly obvious that what kakoune/helix are doing is much better. Iâve also heard enough stories from vim users who tried kakoune and got addicted. But, if you want numbers, it is better than vim at vimgolf: https://news.ycombinator.com/item?id=29984176
Why do that? To be honest, for me the biggest reason is aesthetics: everytime someone opens an .html file in a text editor, html gets parsed with regexes, and thatâs just horrible. See why an IDE for the list of most impactful things semantic understanding of the language provides.
Every project in whatever language it is implemented is âhackableâ and all programming languages are âcollaboration-friendlyâ. Iâm curious what you think about this.
I do think that there are various degrees of collaboration-friendliness. On this axis, I think Rust (and Go as well, though I donât have direct experience with that) is ahead of the average by a wide margin. One significant contribution is tooling: git clone $rust_repo && cd $rust_repo && cargo test works for 90% of rust projects without extra setup. For most other languages, it is âgo read README to learn how we do things here, and then spend some time creating a non-hermetic dev environmentâ. The second significant contribution is that Rust is much better at interfaces (as âsoftware componentsâ, not as âdynamic dispatchâ). This is most visible if we compare memory management between Rust and other sans-GC languages. In something like C++, every codebase has certain rules for lifecycles of objects, and its up to the programmer to ensure that distant parts of code agree on who frees stuff and when. This is a very significant chunk of project-specific knowledge, and is a barrier for contribution. In Rust, there are also project-specific rules about lifetimes, but then the compiler just tell you about them, you donât need to bug the maintainer to learn how things work. But this goes beyond just memory: module&crate system is build around library as a first-class concept, different from a module, which helps tremendously with slicing large projects into independent chunks, and keeping the separation in place throughout refactors.
On a more anecdotal note, IntelliJ Rust and rust-analyzer are similarly very contribution friendly projects, but setting that up with Rust was loads easier than with Kotlin, as gradle is a trainwreck.
It doesnât matter very much what editor I use, because I try to view the problem from an objective perspective rather than entering the war on text editors. If we fight over which editing style/mode is better based on user preferences then what we are doing is politics, not science.
Complexity and performance are crucial aspects in developing software. It didnât work âperfectly fineâ because since 2000 a lot of hardware was deprecated to leave space to solutions that did not take into account complexity and performance as their fundamental problems. See Google et al. for reference. The micro-economical factors that you are talking about are there for a reason, i.e. to let everyone use the software on whatever software/hardware stack they want. Aesthetics is a second tier problem in this case.
Your collaboration-friendliness point is interesting. On one hand you can argue that itâs easier for users to âjust useâ the project as-is without making modifications to the project, and on the other hand you have people that want to make modifications to the project and they need to go to the README in order to actually understand how the project works under the hood. It depends very much on who is your audience. But I would argue that the idea of open-source works better with the second approach.
Second, I would be interested in the complexity of implementing such a code editor. My guess is that it needs to ârecompileâ the code every time you make a change so that it can âreasonâ about your changes.
Helix uses tree-sitter under the hood, which is a fairly new project designed to enable incremental, fallible parsing. Basically, on every keystroke, Helix (through tree-sitter) efficiently updates the syntax tree it has parsed. Thereâs no compilation involved. Edits and selections work on ranges of text that represent pieces of syntax, like a whole function.
more out-of-the-box: default config is reasonable, more features are built-in, editor tries to help the user by providing more introspective contextual help on how to use the thing.
Third, I donât understand your point here. Why more is better?
More is better because it lowers the barrier to entry for picking up the editor. I personally donât enjoy spending a long time figuring out what Vim plugins to use and developing a config file hundreds of lines long just to get reasonable defaults that come standard with, e.g. VSCode and now Helix. My Helix config file is like four or five lines of TOML that change two config options.
Say I need a ordered list of things. Multiple actors (threads, coroutines, whatever) will hold references to things and the list, and will add new entries either before or after the entry they hold. The insertions must be fast and avoid memory copy. A different type of actor will periodically read the list, so order must be maintained. It has to all be in memory, and in process, because latency must be minimum.
What would the rust-native solution for this?
Update: just to clarify, I donât have a actual problem that matches to this constraints, I just tried to come up with something that would be optimally solved with linked lists. Just a thought experiment =)
Could you describe a problem that leads to this sort of set of requirements? My instinct upon hearing this is to dig deeper and see if the problem admits an entirely different kind of solution.
At that point youâre just begging the question. âGiven this problem thatâs been specifically designed to cater for solution X and only solution X, what other solutions are there?â
Well, itâs not only solvable with linked lists, people came up with several interesting possibilities.
I didnât want to âmake linked lists winâ, I just wanted to see if they could be beaten on their on turf, so to speak, and looks like they can, at least sometimes.
Iâd personally use the im crate and use its immutable vectors, and have the threads send new vectors to each other on update. Having multiple writers holding references to the same structure is kinda cursed no matter what youâre doing.
I was thinking something similar. You could even combine this with arc-swap so you could have more precise control over the latency of receiving updates to that data structure.
Alternatively, if your question is about alternatives to linked lists in general, Iâve had success writing code involving multiple concurrent sequenced writers using (1), a ring buffer of pointers, (2) a deque with indexes (instead of pointers), and (3) a hash map of vectors. Thatâs a much bigger problem space than what the articleâs topic, which is (IMO mostly just griping) about people writing libraries that the author thinks arenât necessary.
I donât know how to solve this in concurrent case, but for a single-threaded case one example that comes to mind is the mutable flavor of rust-analyzer syntax trees (for refactors mutable api,where you hold several âcursorsâ into the tree and mutate around them, is most natural). We just do linked list there: https://github.com/rust-analyzer/rowan/blob/master/src/sll.rs.
Even in this obviously-contrived scenario to favor a linked list, I wonder whether this âperiodic low-latencyâ access wonât actually be relatively costly and high latency due to pointer chasing and randomly touching cache lines belonging to different cores. Those atomic pointer swaps will be chatty, and youâll be paying alloc per object.
In modern CPUs there is ridiculously large cost difference between local CPU work and a pipeline stall due to touching cross-core caches and going all the way to the RAM. This is why we have hyperthreading â the RAM-waiting core is stalled for so long that it has time to switch to another thread, do some work there, and switch back before it hears back from RAM.
Iâd try each worker thread append to a non-sorted vector sized appropriately for the cache, then periodically sort it, and send it away to the reader. This amortizes allocations, and is super cache-friendly. The reader then can use merge sort to combine multiple vectors (or iterate without copying). This way data remains owned by one thread most of the time, first sort pass will be multi-threaded but also happen locally in each threadâs cache, and the reader will have mostly-sorted mostly-contiguous snapshot of the data all for itself.
The thread on LKML about this work really doesnât portray the Linux community in a good light. With a dozen or so new kernels being written in Rust, I wouldnât be surprised if this team gives up dealing with Linus and goes to work on adding good Linux ABI compatibility to something else.
I dunno, Linusâ arguments make a lot of sense to me. It sounds like heâs trying to hammer some realism into the idealists. The easter bunny and santa claus comment was a bit much, but otherwise he sounds quite reasonable.
Disagreement is over whether âpanic and stopâ is appropriate for kernel, and here I think Linus is just wrong. Debugging can be done by panic handlers, there is just no need to continue.
Pierre Krieger said it much better, so I will quote:
Part of the reasons why I wrote a kernel is to confirm by experience (as I couldnât be sure before) that âpanic and stopâ is a completely valid way to handle Rust panics even in the kernel, and âwarn and continueâ is extremely harmful. Iâm just so so tired of the defensive programming ideology: âwe canât prevent logic errors therefore the program must be able to continue even a logic error happensâ. Thatâs why my Linux logs are full of stupid warnings that everyone ignores and that everything is buggy.
One argument I can accept is that this should be a separate discussion, and Rust patch should follow Linux rule as it stands, however stupid it may be.
I think the disagreement is more about âshould we have APIs that hide the kernel context from the programmerâ (e.g. âam I in a critical regionâ).
Linusâ writing style has always been kind of hyperbolic/polemic and I donât anticipate that changing :(
But then again Iâm amazed that Rust-in-Linux happened at all, so maybe I should allow for the possibility that Linus will surprise me.
This is exactly what I still donât understand in this discussion. Is there something about stack unwinding and catching the panic that is fundamentally problematic in, eg a driver?
It actually seems like it would be so much better. It recovers some of the resiliency of a microkernel without giving up the performance benefits of a monolithic kernel.
What if, on an irrecoverable error, the graphics driver just panicked, caught the panic at some near-top-level entry point, reset to some known good state and continued? Seems like such an improvement.
I donât believe the Linux kernel has a stack unwinder. I had an intern add one to the FreeBSD kernel a few years ago, but never upstreamed it (*NIX kernel programmers generally donât want it). Kernel stack traces are generated by following frame-pointer chains and are best-effort debugging things, not required for correctness. The Windows kernel has full SEH support and uses it for all sorts of things (for example, if you try to access userspace memory and it faults, you get an exception, whereas in Linux or FreeBSD you use a copy-in or copy-out function to do the access and check the result).
The risk with stack unwinding in a context like this is that the stack unwinder trusts the contents of the stack. If youâre hitting a bug because of stack corruption then the stack unwinder can propagate that corruption elsewhere.
With the objtool/ORC stuff that went into Linux as part of the live-patching work a while back it does actually have a (reliable) stack unwinder: https://lwn.net/Articles/728339/
Thatâs fascinating. Iâm not sure how it actually works for unwinding (rather than walking) the stack: It seems to discard the information about the location of registers other than the stack pointer, so I donât see how it can restore callee-save registers that are spilled to the stack. This is necessary if you want to resume execution (unless you have a setjmp-like mechanism at the catch site, which adds a lot of overhead).
Ah, a terminological misunderstanding then I think â I hadnât realized you meant âunwindingâ specifically as something sophisticated enough to allow resuming execution after popping some number of frames off the stack; I had assumed you just meant traversal of the active frames on the stack, and I think thatâs how the linked article used the term as well (though re-reading your comment now I realize it makes more sense in the way you meant it).
Since AFAIK itâs just to guarantee accurate stack backtraces for determining livepatch safety I donât think the objtool/ORC functionality in the Linux kernel supports unwinding in your sense â I donât know of anything in Linux that would make use of it, aside from maybe userspace memory accesses (though those use a separate âextableâ mechanism for explicitly-marked points in the code that might generate exceptions, e.g. this).
If I understand the userspace access things correctly, they look like the same mechanism as FreeBSD (no stack unwinding, just quick resumption to an error handler if you fault on the access).
I was quite surprised that the ORC[1] is bigger than DWARF. Usually DWARF debug info can get away with being large because itâs stored in separate pages in the binary from the file and so doesnât consume any physical memory unless used. I guess speed does matter for things like DTrace / SystemTap probes, where you want to do a full stack trace quickly, but in the kernel you canât easily lazily load the code.
The NT kernel has some really nice properties here. Almost all of the kernelâs memory (including the kernelâs code) is pageable. This means that the kernelâs unwind metadata can be swapped out if not in use, except for the small bits needed for the page-fault logic. In Windows, the metadata for paged-out pages is stored in PTEs and so you can even page out page-table pages, but you can then potentially need to page in every page in a page-table walk to handle a userspace fault. That extreme case probably mattered a lot more when 16 MiB of RAM was a lot for a workstation than it does now, but being able to page out rarely-used bits of kernel is quite useful.
In addition, the NT kernel has a complete SEH unwinder and so can easily throw exceptions. The SEH exception model is a lot nicer than the Itanium model for in-kernel use. The Itanium C++ ABI allocates exceptions and unwind state on the heap and then does a stack walk, popping frames off to get to handlers. The SEH model allocates them on the stack and then runs each cleanup frame, in turn, on the top of the stack then, at catch, runs some code on top of the stack before popping off all of the remaining frames[2]. This lets you use exceptions to handle out-of-memory conditions (though not out-of-stack-space conditions) reliably.
[1] Such a confusing acronym in this context, given that the modern LLVM JIT is also called ORC.
[2] There are some comments in the SEH code that suggest that itâs flexible enough to support the complete set of Common Lisp exception models, though I donât know if anyone has ever taken advantage of this. The Itanium ABI canât support resumable exceptions and needs some hoop jumping for restartable ones.
What you are missing is that stack unwinding requires destructors, for example to unlock locks you locked. It does work fine for Rust kernels, but not for Linux.
Does the kernel have unprotected memory and just rolls with things like null pointer dereferences reading garbage data?
For errors that are expected Rust uses Result, and in that case itâs easy to sprinkle the code with result.or(whoopsie_fallback) that does not panic.
As far as I understand, yeah, sometimes the kernel would prefer to roll with corrupted memory as far as possible:
So BUG_ON() is basically ALWAYS 100% the wrong thing to do. The
argument that âthere could be memory corruptionâ is [not applicable in this context].
See above why.
As far as I understand, yeah, sometimes the kernel would prefer to roll with corrupted memory as far as possible:
Thatâs what I got from the thread and I donât understand the attitude at all. Once youâve detected memory corruption then there is nothing that a kernel can do safely and anything that it does risks propagating the corruption to persistent storage and destroying the userâs data.
Linus is also wrong that thereâs nothing outside of a kernel that can handle this kind of failure. Modern hardware lets you make it very difficult to accidentally modify the kernel page tables. As I recall, XNU removes all of the pages containing kernel code from the direct map and protects the kernelâs page tables from modification, so that unrecoverable errors can take an interrupt vector to some immutable code that can then write crash dumps or telemetry and reboot. Windows does this from the Secure Kernel, which is effectively a separate VM that has access to all of the main VMâs memory but which is protected from it. On Android, Halfnium provides this kind of abstraction.
I read that entire thread as Linus asserting that the way that Linux does things is the only way that kernel programming can possibly work, ignoring the fact that other kernels use different idioms that are significantly better.
Reading this thread is a little difficult because the discussion is evenly spread between the patch set being proposed, some hypothetical plans for further patch sets, and some existing bad blood between the Linux and Rust community.
The âroll with corrupted memory as far as possibleâ part is probably a case of the âbad bloodâ part. Linux is way more permissive with this than it ought to be but this is probably about something else.
The initial Rust support patch set failed very eagerly and panicked, including on cases where it really is legit not to panic, like when failing to allocate some memory in a driver initialization code. Obviously, the Linux idiom there isnât âgo on with whatever junk pointer kmalloc gives you thereâ â you (hopefully â and this is why we should really root for memory safety, because âhopefullyâ shouldnât be a part of this!) bail out, that driverâs initialization fails but kernel execution obviously continues, as it probably does on just about every general-purpose kernel out there.
The patchsetâs authors actually clarified immediately that the eager panics are actually just an artefact of the early development status â an alloc implementation (and some bits of std) that follows safe kernel idioms was needed, but it was a ton of work so it was scheduled for later, as it really wasnât relevant for a first proof of concept â which was actually a very sane approach.
However, that didnât stop seemingly half the Rustaceans on Twitter to take out their pitchforks, insists that you should absolutely fail hard if memory allocation fails because what else are you going to do, and rant about how Linux is unsafe and itâs riddled with security bugs because itâs written by obsolete monkeys from the nineties whose approach to memory allocation failures is âwell, what could go wrong?â . Which is really not the case, and it really does ignore how much work went into bolting the limited memory safety guarantees that Linux offers on as many systems as it does, while continuing to run critical applications.
So when someone mentions Rustâs safety guarantees, even in hypothetical cases, thereâs a knee-jerk reaction for some folks on the LKML to feel like this is gonna be one of those cases of someone shitting on their work.
I donât want to defend it, itâs absolutely the wrong thing to do and I think experienced developers like Linus should realize thereâs a difference between programmers actually trying to use Rust for real-world problems (like Linux), and Rust advocates for whom everything falls under either âRust excels at thisâ or âthis is an irrelevant niche caseâ. This is not a low-effort patch, lots of thinking went into it, and thereâs bound to be some impedance mismatch between a safe language that tries to offer compile-time guarantees and a kernel historically built on overcoming compiler permisiveness through idioms and well-chosen runtime tradeoffs. I donât think the Linux kernel folks are dealing with this the way they ought to be dealing with it, I just want to offer an interpretation key :-D.
In an ideal world we could have panic and stop in the kernel. But what the kernel does now is what people expect. Itâs very hard to make such a sweeping change.
Sorry, this is a tangent, but your phrasing took me back to one of my favorite webcomics, A Miracle of Science, where mad scientists suffer from a âmemetic diseaseâ that causes them to e.g. monologue and explain their plans (and other cliches), but also allows them to make impossible scientific breakthroughs.
One sign that someone may be suffering from Science Related Memetic Disorder is the phrase âin a perfect worldâ. Itâs never clearly stated exactly why mad scientists tend to say this, but Iâd speculate itâs because in their pursuit of their utopian visions, they make compromises (ethical, ugly hacks to technology, etc.), that they wouldnât have to make in âa perfect worldâ, and this annoys them. Perhaps it drives them to take over the world and make things âperfectâ.
Maybe Iâm missing some context, but it looks like Linus is replying to âwe donât want to invoke undefined behaviorâ with âpanicking is badâ, which makes it seem like irrelevant grandstanding.
The part about debugging specifically makes sense in the âculturalâ context of Linux, but itâs not a matter of realism. There were several attempts to get ârealâ in-kernel debugging support in Linux. None of them really gained much traction, because none of them really worked (as in, reliably, for enough people, and without involving ritual sacrifices), so people sort of begrudgingly settled for debugging by printf and logging unless you really canât do it otherwise. Realistically, there are kernels that do âpanic and stopâ well and are very debuggable.
Also realistically, though: Linux is not one of those kernels, and it doesnât quite have the right architecture for it, either, so backporting one of these approaches onto it is unlikely to be practical. Linusâ arguments are correct in this context but only insofar as they apply to Linux, this isnât a case of hammering realism into idealists. The idealists didnât divine this thing in some programming class that only used pen, paper and algebra, they saw other operating systems doing it.
That being said, I do think people in the Rust advocacy circles really underestimate how difficult it is to get this working well for a production kernel. Implementing panic handling and a barebones in-kernel debugger that can nonetheless usefully handle 99% of the crashes in a tiny microkernel is something you can walk third-year students through. Implementing a useful in-kernel debugger that can reliably debug failures in any context, on NUMA hardware of various architectures, even on a tiny, elegant microkernel, is a whole other story. Pointing out that there are Rust kernels that do it well (Redshirt comes to mind) isnât very productive. I suspect most people already know itâs possible, since e.g. Solaris did it well, years ago. But the kind of work that went into that, on every level of the kernel, not just the debugging end, is mind-blowing.
(Edit: I also suspect this is the usual Rust cultural barrier at work here. The Linux kernel community is absolutely bad at welcoming new contributors. New Rust contributors are also really bad at making themselves welcome. Entertaining the remote theoretical possibility that, unlikely though it might be, it is nonetheless in the realm of physical possibility that you may have to bend your technology around some problems, rather than bending the problems around your technology, or even, God forbid, that you might be wrong about something, can take you a very long way outside a fan bulletin board.)
This sounds like itâs fulfilling the same use case as cargo-crev. Are there fundamental architectural differences between the two tools? Will âreviewsâ from one cross-pollinate to the other?
The linked page proposes some good ideas for how to make the audit process simpler. Especially âdeferred auditsâ. I just wonder why these ideas are implemented in a new crate instead of being upstreamed to cargo-crev.
AFAIU, the focus is on having your own source of trusted audits in tree and not rely on a public database like cargo-crev. But I need to admit that my understanding of both approaches is quite limited.
I really enjoy when someone comes up with a solution Iâve just never even come close to thinking of. The surprise is fun.
Here, it was the realization that we can try to solve the âupdating non-optional fieldsâ challenge by treating readers and writers separately. Time will tell if thatâs actually a good idea, but itâs at least a new point in the design space. Very cool.
Also this library seems surprisingly well fleshed-out. I expected more alpha-quality software, but it seems like everythingâs in order. Great documentation.
I wonder how the performance stacks up against other encoding frameworks. It would be cool to see it added to this comparison benchmark:
Still looking for comprehensive literature on this. I heard that Designing Data-Intensive Applications covers Avro, but Iâd like to see some coverage comparing more options.
Iâve been doing this in Postgres and itâs worked out well. We had a source of JSON events and wanted to store them for analysis, but didnât know ahead of time what indices weâd need. We just knew weâd probably want the ability to do that analysis in the future.
So we ingested everything as jsonb and now I can add generated columns and indices retroactively where needed for query performance. So far, no regrets.
Indeed, the fact that both sqlite and postgres support virtual columns is great that it allows a path for minimal scaling up from the former to the later if desired.
The Grep family is one of those tools I always lose competency with after a few months. I only reliably use it for simple searches and exclusions on STDIN.
I can never remember the difference between grep, egrep, fgrep, and how those versions differ between GNU grep and BSD (macOS) grep. Itâs the same with sed - trying to keep invocations straight between the BSD and GNU variants means I remember neither effectively. find to a lesser extent.
I am very, very happy that ripgrep and fd solved searching file contents and file names for me. Is there a similar replacement for sed? Or a more general modern tool for recursive find-replace file editing on a directory tree? That task is often the sole reason I open an IDE.
I never use egrep or fgrep. From my perspective, these look like relics of the past that almost nobody usea anymore.
Grep does search into multiple files and has support for PCRE.
But yeah, there are BSD and macOS versions of grep that share the name but their utility and performance are not to be compared. They donât even have support for PCRE, which I use 50% of the time.
I used a few of these crappy hubs over the years and always had problems. Tried a Lenovo Thunderbolt 3 dock as mentioned by another commenter, but maybe the one I had was bad because I had issues with it as well.
Finally gave up and bought the CalDigit TS3+ for a ridiculous amount of money (even used off of ebay) and Iâve had zero problems since. This thing is rated one of the best for a reason and now that I never have to mess with these again Iâm happy I spent the money.
Unfortunately if you want a portable option, the CalDigit TS3+ does not apply. Itâs solid enough that Iâd consider their other devices though.
Iâve been looking into docs recently, and donât know much about them. Should I be considering Linux compatibility? Do you know if the CalDigit TS3/4 specifically is compatible with Linux?
I can only speak to the the TS3. It works just as well on my Linux Thinkpad as it does with a Mac. I swap back and forth almost daily with no issues. Iâve used a few distros but am settled on Arch.
Rustâs declarative macros (âmacro_rulesâ) seem pretty similar. Theyâre a bit more verbose, sure, and a little harder to read maybe. But fundamentally similar in that theyâre declarative.
Procedural macros, where you write arbitrary code to produce an AST from an AST, are definitely more difficultâbut also more powerful.
proc macros are token -> token conversions instead of AST -> AST conversions, itâs one of the reasons you need quote/syn for most macro crates. The rfc explains the reasoning here, the big one being so changes to the AST donât break procedural macros.
The core message of the blog post - that you can get away with the worst kind of password if youâre using hardware MFA - is right on the money.
But! Password re-use is what lets people log into all of your (non-MFA) accounts after finding one password in a hacked dump from 2013 that theyâve been chewing on for as long as it takes. Some online vendors probably still arenât salting passwords correctly, making password recovery almost trivial with modern hardware / usual end user password lengths. It wonât show up in these password scan analyses, but can be far more devastating: I still think itâs good advice to not re-use passwords. Write them down in a book, use a password manager, whatever.
& to reiterate that core message: using a 2FA hardware key is the single most effective thing you can do to secure your online accounts. Even if you canât be bothered with 2FA on most of your accounts (which is fair enough) using it on your primary email account will make a huge, huge difference. Anyone with access to the email you used to create other accounts can use that email to reset every account you used it for, so itâs your most important online identity.
The cheap Yubico keys are ÂŁ25 or so. How much will a compromise of your Gmail (Outlook / whatever) cost you?
On one hand, using a Yubikey (or other FIDO device) is extremely easy. Plug it in, touch it. The end. We really just need to expand server-side support.
On the other hand, how do people keep their hardware key always on them? I find this essentially impossible.
Do you carry your keyring around your house/apartment? Maybe my problem is that Iâm unbearably lazy and hate having to get up and walk over to go get my Yubikey from across the room
If youâre using a YubiKey to log into a web site, the odds are that itâs using WebAuthn these days. You donât need a separate token with WebAuthn, you can use the TPM to protect your credentials and have the OS unlock them with biometrics. Most of the works stuff I use is like this: I tough a fingerprint reader on my machine to log in. I might not have my keys with me, but I definitely do have a computer with me when I want to log in. When I want to log in from another machine, my phone can be used to authorise a new device (either registering a new private key or temporarily) by sending a notification to my phone that requires the same process (touch the fingerprint reader and then approve) to log in.
Working on a new markup language for prose (working name: Sona). Example here. Thereâs no parser for this latest version. Still trying to nail down the syntax. Still trying to get rid of the braces. Canât seem to figure out how and still preserve clear hierarchy in a sane way.
The article claims that Rust doesnât have seamless structured concurrency. Rayon (and crossbeamâs scoped threads under the hood) solves most of this problem.
We can use its trait implementations:
use rayon::iter::*;
Define some expensive functions:
/// A number p is prime if iff: (p-1)! mod p = p-1
/// (apparently this is called "Wilson's theorem")
fn is_prime(p: u64) -> bool {
factorial_mod_n(p - 1, p) == p - 1
}
fn factorial_mod_n(k: u64, n: u64) -> u64 {
(1..=k).reduce(|acc, item| (acc * item) % n).unwrap()
}
And then pretty quickly max out every core on your computer finding primes.
fn main() {
// Find primes up to N.
const MAX: u64 = 200000;
// This works.
let primes: Vec<u64> = (2..=MAX).into_par_iter().filter(|&n| is_prime(n)).collect();
}
We can even reference data on the stack. This works because crossbeamâs âscoped threadsâ library enforces that the thread joins back into its parent thread before the reference becomes invalid.
let my_cool_prime = 503;
primes.as_slice().par_iter().for_each(|&prime| {
if prime == my_cool_prime {
println!("found my prime!");
}
});
As mentioned in sidenote 10, one can only sometimes reference data on the stack in parent scopes, only things that happen to be Sync. One will often find that their data doesnât implement Sync, because of some trait object buried within. In that situation, one has to refactor to make it Sync somehow.
If one has to refactor existing code, itâs not seamless, IMO.
Something not being Sync usually means it uses interior mutability without a mutex. Itâs pretty hard to make something magically have a mutex in it; even if you can, itâs a 50/50 chance that itâll be heavily contested and youâll need to restructure it anyway. The other main time Iâve had something not be Sync is when it contains an OS resource or thread-local or such that is explicitly not allowed to be shared between threads, and you generally canât fix that just by find-and-replacing it with Mutex<T>.
Sometimes complicated things are just complicated, I guess.
See the Rust example linked from the article (I think it was in a side note), something as simple as having a regular trait object somewhere in your data is enough to make something not Sync.
Youâve complained that non-Sync (i.e. non-thread-safe) trait objects are prevented from being used in multi-threaded code. If they were allowed, that would be unsound, and âseamlesslyâ allow data race bugs.
Just to be clear: Rust trait objects can be compatible with multi-threaded code. You donât need to eliminate them or work around them. They just come in two flavors: thread-safe and non-thread-safe. If you have a non-thread-safe trait object, then of course you canât âseamlesslyâ use it, as that would be incorrect. But when you have dyn Trait + Send + Sync, it just works in both single-threaded and multi-threaded contexts.
Thatâs a pretty big misinterpretation of my words. Iâm happy to clarify what Iâve said.
I am not suggesting any changes to Rust to make safe seamless concurrency possible; Iâm not sure thatâs even possible, without introducing a lot of complexity to the language. Iâm not saying it should change its rules to allow use of non-Sync/non-Send data in multi-threaded code. Iâm not saying anything at all about changing Rust.
Iâm simply stating that, because Rust requires refactoring existing code (for example deeply changing trait objects and other existing data to be Sync), it cannot do this seamless concurrency. If youâre really curious about the root cause of why Rust requires Sync and Send, and therefore why it canât do seamless concurrency, let me know and I can enlighten. All I mean so far is that Rust cannot do this, because of its own decisions and limitations.
But by this definition, languages that care about correctness of multi-threaded code canât be âseamlessâ. Because otherwise you can always be in a situation where youâre dealing with code that either actually isnât thread-safe, or incorrectly declares itself to be thread-unsafe, and you need to fix it instead of having compiler ignore the problem and let you run it anyway. From Rustâs perspective the refactoring youâre describing is a bug fix, because this situation would not happen in properly written code.
Thatâs not true; the article shows an example where a language can safely have multiple threads concurrently reading the same data, without refactoring.
It also links to an example showing that it is not possible to do safely in Rust, without refactoring. Given that it is possible, but not in Rust, it indicates that the problem is with Rust here.
From Rustâs perspective the refactoring youâre describing is a bug fix, because this situation would not happen in properly written code.
Yes, if we tried this in Rust, it would be a bug in that Rust code, because Rust canât safely express this pattern. That doesnât mean it would be a bug in other languages.
Youâre disqualifying Rust for not being able to safely express this pattern? But you include OpenMP, which doesnât even have these guarantees in the first place? How is that not favoring lack of safety checks as seamlessness?
I see youâre bringing up OpenMP suddenly, when our conversation so far hasnât been comparing C to Rust, it has been about safe languages.
Perhaps our conversationâs context didnât make it clear enough, so Iâll be more specific: Rust cannot support seamless fearless structured concurrency, but the article shows that we can have seamless fearless structured concurrency in another language, such as Vale.
Iâm sure youâre discussing in good faith, and not intentionally straw-manning what Iâm saying, so Iâm happy to keep clarifying. I always enjoy exploring whatâs possible in language design with those who are honest and curious.
Iâm just contesting what can be called âseamlessâ, which is more of argument about semantics of the word, and tangential to the Vale language, so itâs probably not worth digging this further. Thank you for your responses.
Yeah, it would be interesting to have trait objects follow the same rules as deriving traits, same way that new types are automatically Send+Sync+Drop when they can be unless you specify otherwise. ie, saying dyn Trait usually means dyn Trait+Send+Sync+Drop. It would probably be a breaking change to do that, so will probably never happen, but I wonder whether itâs one of those things would abruptly spiral out of control and make everything in the world break, or whether it would be a minor inconvenience to implement and then mostly just work everywhere.
DNS is full of confusing jargon. At $work we do R&D closely related to DNS and run into this all the time.
Example: what is a âbase domain nameâ? If you go by ICANN, the BDN for âalice.github.ioâ is âioâ, but if you go by the Public Suffix List, the âbase domainâ (more often called a âsiteâ in this context), is âgithub.ioâ because that definition is more relavent for security and privacy on The Web.
A more accurate name that someone was kind enough to invent is TLD+1 (vs eTLD+1) so itâs clear which we mean. Iâve seen âBDNâ refer to both, but more commonly TLD+1.
But somehow, in the midst of all this jargon, we still donât have a reliable way to refer to the â+1â label in TLD+1, even though itâs the most important label in the group. In âapple.comâ, what is âappleâ? At work we call it the âprimary name labelâ, though that phrase also misses the subtlety of distinguishing between TLDs and eTLDs, and isnât widely used as far as I know. (If someone knows official terminology for this, please tell me.)
And then thereâs the hostname/FQDN/domain triplet that all pretty much mean the same thing (until they donât).
And then thereâs the difference between a DNS proxy (sometimes called a forwarding resolver) and a DNS server, which is fairly close to the point made in the article. As far as I can tell, the difference is in whether or not the server caches responses. Or at least thatâs where I think the difference should be. Some people distinguish between proxies and resolvers. Others donât.
Also âlabelâ vs âsubdomainâ. Thatâs a good one. Is the âroot domainâ a subdomain? Itâs usually considered to be an empty label. But if root is a subdomain, shouldnât it be a subdomain of something?
Thereâs more fun in DNS (think about how domains are supposed to be case-insensitive in a Unicode world (also domains arenât necessarily Unicode and can be arbitrary bytes)), but Iâll leave it there before I get too much further off-topic.
You had me in the first half, not gonna lie!
Elixir is made up of very delightful layers of functionality. One time I wanted to submit a proposal to make the pipeline operator more general. I wanted to be able to pass in an underscore so you could specify where the piped argument would go. Something like
1 |> subtract(2, _). I was able to download the repo, find the code I needed to change and fix up the test cases in like an hour or two. It was very readable and easy to build. Would recommend checking it out.There is built in way for that using
Kernel.then/2function in form1 |> then(&subtract(2, &1)). I also have createdmagrittelibrary that extend|>to support1 |> subtract(2, ...)form.Magritte, such a good name for a
pipelibrary đThe library tagline on Hex is (of course):
Thatâs awesome! I didnât think of replacing the operator. I figured that wouldnât be an option.
Iâve had that same operator idea since forever. The restriction on where the argument goes with the pipe is relatively easy to work around but never feels general enough.
Weâre in good company! There is a SFRI that does this for scheme:
https://srfi.schemers.org/srfi-197/srfi-197.html
I had that same pipe/underscore idea! It seems so arbitrary that the first argument is always the one filled in by the pipe.
Are there any papers or posts that describe the current Pijul data model? Iâd be interested in reading about the underlying format. I know itâs undergone some revisions over the past few years.
No, not really. The underlying format is complicated, mostly because it relies on Sanakirja, which is itself complicated.
What is the reason to not
def send_emails(list_of_recipients, bccs=[]):?From my Clojure backend I wonder what is sending the emails (and how you can possibly test that unless you are passing in something that will send the email), but perhaps there is something I miss from the Python perspective?
Hi! Thanks for reading! đ
The answer is mutable default arguments; if youâre coming from Clojure, I can see why this wouldnât be an issue đ
When Python is defining this function, itâs making the default argument a single instance of an empty list, not âa fresh empty list created when invoking the function.â So if that function mutates the parameter in some way, that mutation will persist across calls, which leads to buggy behavior.
An example might be (with a whole module):
What happens is: every time you call it without specifying a second argument, the emails added from previous invocations will still be in the default
bccslist.The way to actually do a âdefault to empty listâ in Python is:
This is horrifying. TIL, but I almost wish I hadnât.
It makes sense outside of the pass-by-reference/pass-by-value dichotomy but itâs still a massive footgun.
As for how you test it, itâs probably with mocking. Python has introspection/monkey patching abilities, so if you relied on another library to handle actually sending the email (the example above I pretended
sendgridhad an SDK with a function calledactually_send_email, in Python you would usually do something likeThis mucks with the module at runtime to convert
sendgrid.actually_send_emailto instead record its calls instead of what the function did originally.docs here
Itâs not about sending emails, but the mutable default argument: https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments
Python will initialize the function argument variables on module import, rather than whenever the function is called.
The embedded computation looks like a trap which will make papers harder to reproduce. Is there support for directly embedding scripts from other languages?
There are built-in ways to use JSON data. I think for complex or expensive computations, users could leverage their preferred language, output JSON, and then pick it up as input in Typst.
I donât know, I hope so, and someone has asked at https://github.com/typst/typst/issues/117
Whatâs the editor support like? I assume the best thing to do would be create a tree-sitter grammar for this, unless it already exists?
Not an answer to your question, but related: they also develop an online collaborative editor that looks very interesting: https://typst.app/
I suspect that the moment Helix supports this format, I will start converting all my notes from Markdown.
I absolutely love the look of this, it seems to be everything Iâve dreamt of in a LaTeX replacement. The web UI is really nice too, although it seems to be struggling a bit under the load post launch announcement.
Are you writing lots of markdown in Helix? Whenever I try, I get annoyed by the lack of soft wrap. I suspect Iâd have the same problem writing LaTeX or Typst with Helix.
Helix has soft wrap in HEAD since #5893 was merged two weeks ago. If you run nightly, just add
editor.soft-wrap.enable = trueto your config. Else of course wait for the next release, though I donât know if there are plans yet for when it will drop.Oh, and according to (nightly)
hx --health, Helix doesnât support Typst yet. I guess it will be added quickly once there is a tree sitter for it.Oh wow, thatâs great news! This could definitely change my workflow.
Personally I always use
:reflowor pipe tofmtto hard wrap lines when I write Markdown. But itâs also good to hear that soft wrap is coming soon!Nonexistent :â]
Honestly the syntax is not the worst thing to write without highlighting, but Iâd love an $EDITOR plugin.
Itâs weird to see a Rust UI post that doesnât at least mention Druid and related projects. Raph Levien and others have spent a long time thinking about why Rust GUI is âhardâ and the right data model to tackle the problem.
Raphâs âErgonomic APIs for hard problemsâ keynote at RustLab 2022 is also worth a look if you havenât seen it (recording, slides).
To me itâs not very clear what problem does Helix try to solve and what are the shortcomings of Vi/Vim/Neovim for solving that problem? I admit that I did not read very carefully the documentation of this project but I was expecting to get that information from the front page of the project.
I think the four axis of improvement are:
Sorry but Iâm not convinced that those are real improvements. Letâs take them one by one. First, I see no numbers to hold the claim that the new model is better than the old model. The thing that they give as an example in the link can be done in Vim with Visual Mode, more or less. So I would take it as a proposal for improvement, not as an actual improvement. Second, I would be interested in the complexity of implementing such a code editor. My guess is that it needs to ârecompileâ the code every time you make a change so that it can âreasonâ about your changes. This looks to me as an overkill for large files or large projects. More than that, old systems cannot afford this type of ârecompilationâ. Third, I donât understand your point here. Why more is better? Fourth, call me unagreeable, but I donât buy the Rust propaganda. Every project in whatever language it is implemented is âhackableâ and all programming languages are âcollaboration-friendlyâ. Iâm curious what you think about this.
I am not trying to convince you, you do sound like a person quite happy with vi, and thatâs perfectly fine! It feels like helix solves the problems you personally donât have. But I am pretty sure these are real problems for quite a few people (and I personally definitely acutely feel all of them).
Re editing model: I personally wouldnât rely on numbers to make a decision here. To me, itâs blindingly obvious that what kakoune/helix are doing is much better. Iâve also heard enough stories from vim users who tried kakoune and got addicted. But, if you want numbers, it is better than vim at vimgolf: https://news.ycombinator.com/item?id=29984176
Re code editor complexity&performance: weâve been doing that since early 2000âs in Java perfectly fine, there are no insurmountable technical or performance challenges there. Itâs just due to open source micro economical reasons that no one actually bothered to go beyond regexes until Microsoft did the thing (see why LSP). As another data point, I started solving IDE problem for Rust in 2015, and by 2021 it was thoroughly solved, twice (by IntelliJ Rust and rust-analyzer). Some docs for that are in https://github.com/rust-lang/rust-analyzer/blob/master/docs/dev/architecture.md, https://rust-analyzer.github.io/blog, https://www.youtube.com/playlist?list=PLhb66M_x9UmrqXhQuIpWC5VgTdrGxMx3y)
Why do that? To be honest, for me the biggest reason is aesthetics: everytime someone opens an
.htmlfile in a text editor, html gets parsed with regexes, and thatâs just horrible. See why an IDE for the list of most impactful things semantic understanding of the language provides.I do think that there are various degrees of collaboration-friendliness. On this axis, I think Rust (and Go as well, though I donât have direct experience with that) is ahead of the average by a wide margin. One significant contribution is tooling:
git clone $rust_repo && cd $rust_repo && cargo testworks for 90% of rust projects without extra setup. For most other languages, it is âgo read README to learn how we do things here, and then spend some time creating a non-hermetic dev environmentâ. The second significant contribution is that Rust is much better at interfaces (as âsoftware componentsâ, not as âdynamic dispatchâ). This is most visible if we compare memory management between Rust and other sans-GC languages. In something like C++, every codebase has certain rules for lifecycles of objects, and its up to the programmer to ensure that distant parts of code agree on who frees stuff and when. This is a very significant chunk of project-specific knowledge, and is a barrier for contribution. In Rust, there are also project-specific rules about lifetimes, but then the compiler just tell you about them, you donât need to bug the maintainer to learn how things work. But this goes beyond just memory: module&crate system is build around library as a first-class concept, different from a module, which helps tremendously with slicing large projects into independent chunks, and keeping the separation in place throughout refactors.On a more anecdotal note, IntelliJ Rust and rust-analyzer are similarly very contribution friendly projects, but setting that up with Rust was loads easier than with Kotlin, as gradle is a trainwreck.
It doesnât matter very much what editor I use, because I try to view the problem from an objective perspective rather than entering the war on text editors. If we fight over which editing style/mode is better based on user preferences then what we are doing is politics, not science.
Complexity and performance are crucial aspects in developing software. It didnât work âperfectly fineâ because since 2000 a lot of hardware was deprecated to leave space to solutions that did not take into account complexity and performance as their fundamental problems. See Google et al. for reference. The micro-economical factors that you are talking about are there for a reason, i.e. to let everyone use the software on whatever software/hardware stack they want. Aesthetics is a second tier problem in this case.
Your collaboration-friendliness point is interesting. On one hand you can argue that itâs easier for users to âjust useâ the project as-is without making modifications to the project, and on the other hand you have people that want to make modifications to the project and they need to go to the README in order to actually understand how the project works under the hood. It depends very much on who is your audience. But I would argue that the idea of open-source works better with the second approach.
Helix uses tree-sitter under the hood, which is a fairly new project designed to enable incremental, fallible parsing. Basically, on every keystroke, Helix (through tree-sitter) efficiently updates the syntax tree it has parsed. Thereâs no compilation involved. Edits and selections work on ranges of text that represent pieces of syntax, like a whole function.
More is better because it lowers the barrier to entry for picking up the editor. I personally donât enjoy spending a long time figuring out what Vim plugins to use and developing a config file hundreds of lines long just to get reasonable defaults that come standard with, e.g. VSCode and now Helix. My Helix config file is like four or five lines of TOML that change two config options.
Out of curiosity:
Say I need a ordered list of things. Multiple actors (threads, coroutines, whatever) will hold references to things and the list, and will add new entries either before or after the entry they hold. The insertions must be fast and avoid memory copy. A different type of actor will periodically read the list, so order must be maintained. It has to all be in memory, and in process, because latency must be minimum.
What would the rust-native solution for this?
Update: just to clarify, I donât have a actual problem that matches to this constraints, I just tried to come up with something that would be optimally solved with linked lists. Just a thought experiment =)
Could you describe a problem that leads to this sort of set of requirements? My instinct upon hearing this is to dig deeper and see if the problem admits an entirely different kind of solution.
Oh, there is no problem, I just tried to come up with a set of constraints that would be best suited to be solved with a linked list.
Each actor has a local sorted vector. The reader thread iterates over them for a total ordered view.
Why three?
Oh, I said three because of the three items in multiple actors. One vector per thread. Iâll edit to clarify.
At that point youâre just begging the question. âGiven this problem thatâs been specifically designed to cater for solution X and only solution X, what other solutions are there?â
Well, itâs not only solvable with linked lists, people came up with several interesting possibilities.
I didnât want to âmake linked lists winâ, I just wanted to see if they could be beaten on their on turf, so to speak, and looks like they can, at least sometimes.
How many things? How large are they? Are they comparable for order or is this insertion order only?
Avoiding a memory copy implies chasing pointers which implies latency.
Echoing @justinpombrio, whatâs the problem weâre solving?
Iâd personally use the
imcrate and use its immutable vectors, and have the threads send new vectors to each other on update. Having multiple writers holding references to the same structure is kinda cursed no matter what youâre doing.I was thinking something similar. You could even combine this with arc-swap so you could have more precise control over the latency of receiving updates to that data structure.
The most idiomatic Rust data structure for linked lists is
std::collections::LinkedList.Alternatively, if your question is about alternatives to linked lists in general, Iâve had success writing code involving multiple concurrent sequenced writers using (1), a ring buffer of pointers, (2) a deque with indexes (instead of pointers), and (3) a hash map of vectors. Thatâs a much bigger problem space than what the articleâs topic, which is (IMO mostly just griping) about people writing libraries that the author thinks arenât necessary.
std::LinkedListdoesnât have cursors on stable (and doesnât have multiple cursors on nightly), so it doesnât solve the problem.I donât know how to solve this in concurrent case, but for a single-threaded case one example that comes to mind is the mutable flavor of rust-analyzer syntax trees (for refactors mutable api,where you hold several âcursorsâ into the tree and mutate around them, is most natural). We just do linked list there: https://github.com/rust-analyzer/rowan/blob/master/src/sll.rs.
Even in this obviously-contrived scenario to favor a linked list, I wonder whether this âperiodic low-latencyâ access wonât actually be relatively costly and high latency due to pointer chasing and randomly touching cache lines belonging to different cores. Those atomic pointer swaps will be chatty, and youâll be paying alloc per object.
In modern CPUs there is ridiculously large cost difference between local CPU work and a pipeline stall due to touching cross-core caches and going all the way to the RAM. This is why we have hyperthreading â the RAM-waiting core is stalled for so long that it has time to switch to another thread, do some work there, and switch back before it hears back from RAM.
Iâd try each worker thread append to a non-sorted vector sized appropriately for the cache, then periodically sort it, and send it away to the reader. This amortizes allocations, and is super cache-friendly. The reader then can use merge sort to combine multiple vectors (or iterate without copying). This way data remains owned by one thread most of the time, first sort pass will be multi-threaded but also happen locally in each threadâs cache, and the reader will have mostly-sorted mostly-contiguous snapshot of the data all for itself.
You might not believe me, but I swear cache did cross my mind while I was writing my, indeed, very and purposely contrived example XD
All these answers are almost making me wonder where linked lists are good for anything anymore, really.
I am fairly confident that liked list are the most overused data structure out there, but it doesnât mean that they are entirely useless.
Off the top of my head:
You can have âfatâ linked lists analogous to cdr coding wherein each node has a fixed capacity greater than 1 (and a variable occupancy).
The thread on LKML about this work really doesnât portray the Linux community in a good light. With a dozen or so new kernels being written in Rust, I wouldnât be surprised if this team gives up dealing with Linus and goes to work on adding good Linux ABI compatibility to something else.
I dunno, Linusâ arguments make a lot of sense to me. It sounds like heâs trying to hammer some realism into the idealists. The easter bunny and santa claus comment was a bit much, but otherwise he sounds quite reasonable.
Disagreement is over whether âpanic and stopâ is appropriate for kernel, and here I think Linus is just wrong. Debugging can be done by panic handlers, there is just no need to continue.
Pierre Krieger said it much better, so I will quote:
One argument I can accept is that this should be a separate discussion, and Rust patch should follow Linux rule as it stands, however stupid it may be.
I think the disagreement is more about âshould we have APIs that hide the kernel context from the programmerâ (e.g. âam I in a critical regionâ).
This message made some sense to me: https://lkml.org/lkml/2022/9/19/840
Linusâ writing style has always been kind of hyperbolic/polemic and I donât anticipate that changing :( But then again Iâm amazed that Rust-in-Linux happened at all, so maybe I should allow for the possibility that Linus will surprise me.
This is exactly what I still donât understand in this discussion. Is there something about stack unwinding and catching the panic that is fundamentally problematic in, eg a driver?
It actually seems like it would be so much better. It recovers some of the resiliency of a microkernel without giving up the performance benefits of a monolithic kernel.
What if, on an irrecoverable error, the graphics driver just panicked, caught the panic at some near-top-level entry point, reset to some known good state and continued? Seems like such an improvement.
I donât believe the Linux kernel has a stack unwinder. I had an intern add one to the FreeBSD kernel a few years ago, but never upstreamed it (*NIX kernel programmers generally donât want it). Kernel stack traces are generated by following frame-pointer chains and are best-effort debugging things, not required for correctness. The Windows kernel has full SEH support and uses it for all sorts of things (for example, if you try to access userspace memory and it faults, you get an exception, whereas in Linux or FreeBSD you use a copy-in or copy-out function to do the access and check the result).
The risk with stack unwinding in a context like this is that the stack unwinder trusts the contents of the stack. If youâre hitting a bug because of stack corruption then the stack unwinder can propagate that corruption elsewhere.
With the objtool/ORC stuff that went into Linux as part of the live-patching work a while back it does actually have a (reliable) stack unwinder: https://lwn.net/Articles/728339/
Thatâs fascinating. Iâm not sure how it actually works for unwinding (rather than walking) the stack: It seems to discard the information about the location of registers other than the stack pointer, so I donât see how it can restore callee-save registers that are spilled to the stack. This is necessary if you want to resume execution (unless you have a setjmp-like mechanism at the catch site, which adds a lot of overhead).
Ah, a terminological misunderstanding then I think â I hadnât realized you meant âunwindingâ specifically as something sophisticated enough to allow resuming execution after popping some number of frames off the stack; I had assumed you just meant traversal of the active frames on the stack, and I think thatâs how the linked article used the term as well (though re-reading your comment now I realize it makes more sense in the way you meant it).
Since AFAIK itâs just to guarantee accurate stack backtraces for determining livepatch safety I donât think the objtool/ORC functionality in the Linux kernel supports unwinding in your sense â I donât know of anything in Linux that would make use of it, aside from maybe userspace memory accesses (though those use a separate âextableâ mechanism for explicitly-marked points in the code that might generate exceptions, e.g. this).
If I understand the userspace access things correctly, they look like the same mechanism as FreeBSD (no stack unwinding, just quick resumption to an error handler if you fault on the access).
I was quite surprised that the ORC[1] is bigger than DWARF. Usually DWARF debug info can get away with being large because itâs stored in separate pages in the binary from the file and so doesnât consume any physical memory unless used. I guess speed does matter for things like DTrace / SystemTap probes, where you want to do a full stack trace quickly, but in the kernel you canât easily lazily load the code.
The NT kernel has some really nice properties here. Almost all of the kernelâs memory (including the kernelâs code) is pageable. This means that the kernelâs unwind metadata can be swapped out if not in use, except for the small bits needed for the page-fault logic. In Windows, the metadata for paged-out pages is stored in PTEs and so you can even page out page-table pages, but you can then potentially need to page in every page in a page-table walk to handle a userspace fault. That extreme case probably mattered a lot more when 16 MiB of RAM was a lot for a workstation than it does now, but being able to page out rarely-used bits of kernel is quite useful.
In addition, the NT kernel has a complete SEH unwinder and so can easily throw exceptions. The SEH exception model is a lot nicer than the Itanium model for in-kernel use. The Itanium C++ ABI allocates exceptions and unwind state on the heap and then does a stack walk, popping frames off to get to handlers. The SEH model allocates them on the stack and then runs each cleanup frame, in turn, on the top of the stack then, at catch, runs some code on top of the stack before popping off all of the remaining frames[2]. This lets you use exceptions to handle out-of-memory conditions (though not out-of-stack-space conditions) reliably.
[1] Such a confusing acronym in this context, given that the modern LLVM JIT is also called ORC.
[2] There are some comments in the SEH code that suggest that itâs flexible enough to support the complete set of Common Lisp exception models, though I donât know if anyone has ever taken advantage of this. The Itanium ABI canât support resumable exceptions and needs some hoop jumping for restartable ones.
What you are missing is that stack unwinding requires destructors, for example to unlock locks you locked. It does work fine for Rust kernels, but not for Linux.
Does the kernel have unprotected memory and just rolls with things like null pointer dereferences reading garbage data?
For errors that are expected Rust uses
Result, and in that case itâs easy to sprinkle the code withresult.or(whoopsie_fallback)that does not panic.As far as I understand, yeah, sometimes the kernel would prefer to roll with corrupted memory as far as possible:
(from docs and linked mail).
null derefernces in particular though usually do what BUG_ON essentially does.
And things like out-of-bounds accesses seem to end with null-dereference:
https://github.com/torvalds/linux/blob/45b588d6e5cc172704bac0c998ce54873b149b22/lib/flex_array.c#L268-L269
Though, notably, out-of-bounds access doesnât immediately crash the thing.
Thatâs what I got from the thread and I donât understand the attitude at all. Once youâve detected memory corruption then there is nothing that a kernel can do safely and anything that it does risks propagating the corruption to persistent storage and destroying the userâs data.
Linus is also wrong that thereâs nothing outside of a kernel that can handle this kind of failure. Modern hardware lets you make it very difficult to accidentally modify the kernel page tables. As I recall, XNU removes all of the pages containing kernel code from the direct map and protects the kernelâs page tables from modification, so that unrecoverable errors can take an interrupt vector to some immutable code that can then write crash dumps or telemetry and reboot. Windows does this from the Secure Kernel, which is effectively a separate VM that has access to all of the main VMâs memory but which is protected from it. On Android, Halfnium provides this kind of abstraction.
I read that entire thread as Linus asserting that the way that Linux does things is the only way that kernel programming can possibly work, ignoring the fact that other kernels use different idioms that are significantly better.
Reading this thread is a little difficult because the discussion is evenly spread between the patch set being proposed, some hypothetical plans for further patch sets, and some existing bad blood between the Linux and Rust community.
The âroll with corrupted memory as far as possibleâ part is probably a case of the âbad bloodâ part. Linux is way more permissive with this than it ought to be but this is probably about something else.
The initial Rust support patch set failed very eagerly and panicked, including on cases where it really is legit not to panic, like when failing to allocate some memory in a driver initialization code. Obviously, the Linux idiom there isnât âgo on with whatever junk pointer
kmallocgives you thereâ â you (hopefully â and this is why we should really root for memory safety, because âhopefullyâ shouldnât be a part of this!) bail out, that driverâs initialization fails but kernel execution obviously continues, as it probably does on just about every general-purpose kernel out there.The patchsetâs authors actually clarified immediately that the eager panics are actually just an artefact of the early development status â an
allocimplementation (and some bits ofstd) that follows safe kernel idioms was needed, but it was a ton of work so it was scheduled for later, as it really wasnât relevant for a first proof of concept â which was actually a very sane approach.However, that didnât stop seemingly half the Rustaceans on Twitter to take out their pitchforks, insists that you should absolutely fail hard if memory allocation fails because what else are you going to do, and rant about how Linux is unsafe and itâs riddled with security bugs because itâs written by obsolete monkeys from the nineties whose approach to memory allocation failures is âwell, what could go wrong?â . Which is really not the case, and it really does ignore how much work went into bolting the limited memory safety guarantees that Linux offers on as many systems as it does, while continuing to run critical applications.
So when someone mentions Rustâs safety guarantees, even in hypothetical cases, thereâs a knee-jerk reaction for some folks on the LKML to feel like this is gonna be one of those cases of someone shitting on their work.
I donât want to defend it, itâs absolutely the wrong thing to do and I think experienced developers like Linus should realize thereâs a difference between programmers actually trying to use Rust for real-world problems (like Linux), and Rust advocates for whom everything falls under either âRust excels at thisâ or âthis is an irrelevant niche caseâ. This is not a low-effort patch, lots of thinking went into it, and thereâs bound to be some impedance mismatch between a safe language that tries to offer compile-time guarantees and a kernel historically built on overcoming compiler permisiveness through idioms and well-chosen runtime tradeoffs. I donât think the Linux kernel folks are dealing with this the way they ought to be dealing with it, I just want to offer an interpretation key :-D.
No expert here, but I imagine linux kernel has methods of handling expected errors & null checks.
In an ideal world we could have panic and stop in the kernel. But what the kernel does now is what people expect. Itâs very hard to make such a sweeping change.
Sorry, this is a tangent, but your phrasing took me back to one of my favorite webcomics, A Miracle of Science, where mad scientists suffer from a âmemetic diseaseâ that causes them to e.g. monologue and explain their plans (and other cliches), but also allows them to make impossible scientific breakthroughs.
One sign that someone may be suffering from Science Related Memetic Disorder is the phrase âin a perfect worldâ. Itâs never clearly stated exactly why mad scientists tend to say this, but Iâd speculate itâs because in their pursuit of their utopian visions, they make compromises (ethical, ugly hacks to technology, etc.), that they wouldnât have to make in âa perfect worldâ, and this annoys them. Perhaps it drives them to take over the world and make things âperfectâ.
So I have to ask⌠are you a mad scientist?
I aspire to be? bwahahaa
Hah, thanks for introducing me to that comic! I ended up archive-bingeing it.
What modern kernels use âpanic and stopâ? Is it a feature of the BSDs?
Every kernel except Linux.
I didnât exactly mean bsd. And I canât name one. But verified ones? redox?
Iâm sorry if my question came off as curt or snide, I was asking out of genuine ignorance. I donât know much about kernels at this level.
I was wondering how much an outlier the Linux kernel is - @4ad âs comment suggests it is.
No harm done
I agree. I would be very worried if people writing the Linux kernel adopted the âif it compiles it worksâ mindset.
Maybe Iâm missing some context, but it looks like Linus is replying to âwe donât want to invoke undefined behaviorâ with âpanicking is badâ, which makes it seem like irrelevant grandstanding.
The part about debugging specifically makes sense in the âculturalâ context of Linux, but itâs not a matter of realism. There were several attempts to get ârealâ in-kernel debugging support in Linux. None of them really gained much traction, because none of them really worked (as in, reliably, for enough people, and without involving ritual sacrifices), so people sort of begrudgingly settled for debugging by printf and logging unless you really canât do it otherwise. Realistically, there are kernels that do âpanic and stopâ well and are very debuggable.
Also realistically, though: Linux is not one of those kernels, and it doesnât quite have the right architecture for it, either, so backporting one of these approaches onto it is unlikely to be practical. Linusâ arguments are correct in this context but only insofar as they apply to Linux, this isnât a case of hammering realism into idealists. The idealists didnât divine this thing in some programming class that only used pen, paper and algebra, they saw other operating systems doing it.
That being said, I do think people in the Rust advocacy circles really underestimate how difficult it is to get this working well for a production kernel. Implementing panic handling and a barebones in-kernel debugger that can nonetheless usefully handle 99% of the crashes in a tiny microkernel is something you can walk third-year students through. Implementing a useful in-kernel debugger that can reliably debug failures in any context, on NUMA hardware of various architectures, even on a tiny, elegant microkernel, is a whole other story. Pointing out that there are Rust kernels that do it well (Redshirt comes to mind) isnât very productive. I suspect most people already know itâs possible, since e.g. Solaris did it well, years ago. But the kind of work that went into that, on every level of the kernel, not just the debugging end, is mind-blowing.
(Edit: I also suspect this is the usual Rust cultural barrier at work here. The Linux kernel community is absolutely bad at welcoming new contributors. New Rust contributors are also really bad at making themselves welcome. Entertaining the remote theoretical possibility that, unlikely though it might be, it is nonetheless in the realm of physical possibility that you may have to bend your technology around some problems, rather than bending the problems around your technology, or even, God forbid, that you might be wrong about something, can take you a very long way outside a fan bulletin board.)
Wow, Linus really has mellowed over the years ;)
It will be interesting to see how
cargo vetfits into this space once itâs announced officially.Previous discussion: here.
This sounds like itâs fulfilling the same use case as cargo-crev. Are there fundamental architectural differences between the two tools? Will âreviewsâ from one cross-pollinate to the other?
The linked page proposes some good ideas for how to make the audit process simpler. Especially âdeferred auditsâ. I just wonder why these ideas are implemented in a new crate instead of being upstreamed to cargo-crev.
AFAIU, the focus is on having your own source of trusted audits in tree and not rely on a public database like cargo-crev. But I need to admit that my understanding of both approaches is quite limited.
I really enjoy when someone comes up with a solution Iâve just never even come close to thinking of. The surprise is fun.
Here, it was the realization that we can try to solve the âupdating non-optional fieldsâ challenge by treating readers and writers separately. Time will tell if thatâs actually a good idea, but itâs at least a new point in the design space. Very cool.
Also this library seems surprisingly well fleshed-out. I expected more alpha-quality software, but it seems like everythingâs in order. Great documentation.
I wonder how the performance stacks up against other encoding frameworks. It would be cool to see it added to this comparison benchmark:
https://github.com/djkoloski/rust_serialization_benchmark
Thanks for the kind words! It was a pleasant surprise waking up and seeing Typical on Lobsters this morning.
I learnt about it on ZuriHac. Tie, which allows generation of Haskell server stubs from OpenAPI was presented there, and given that we at work generate OpenAPI from applicative style Haskell schemas (like autodocodec, but not type class based), I am curious about this space.
Still looking for comprehensive literature on this. I heard that Designing Data-Intensive Applications covers Avro, but Iâd like to see some coverage comparing more options.
I use this a lot in conjunction with a json blob. This allows for document storage without super strict structure enforcement:
Something like:
Iâve been doing this in Postgres and itâs worked out well. We had a source of JSON events and wanted to store them for analysis, but didnât know ahead of time what indices weâd need. We just knew weâd probably want the ability to do that analysis in the future.
So we ingested everything as
jsonband now I can add generated columns and indices retroactively where needed for query performance. So far, no regrets.Indeed, the fact that both sqlite and postgres support virtual columns is great that it allows a path for minimal scaling up from the former to the later if desired.
What is stored if the field is missing?
NULLis stored:The Grep family is one of those tools I always lose competency with after a few months. I only reliably use it for simple searches and exclusions on STDIN.
I can never remember the difference between grep, egrep, fgrep, and how those versions differ between GNU grep and BSD (macOS) grep. Itâs the same with
sed- trying to keep invocations straight between the BSD and GNU variants means I remember neither effectively.findto a lesser extent.I am very, very happy that ripgrep and fd solved searching file contents and file names for me. Is there a similar replacement for
sed? Or a more general modern tool for recursive find-replace file editing on a directory tree? That task is often the sole reason I open an IDE.I never use egrep or fgrep. From my perspective, these look like relics of the past that almost nobody usea anymore.
Grep does search into multiple files and has support for PCRE.
But yeah, there are BSD and macOS versions of grep that share the name but their utility and performance are not to be compared. They donât even have support for PCRE, which I use 50% of the time.
I use sd all the time and I love it.
I used a few of these crappy hubs over the years and always had problems. Tried a Lenovo Thunderbolt 3 dock as mentioned by another commenter, but maybe the one I had was bad because I had issues with it as well.
Finally gave up and bought the CalDigit TS3+ for a ridiculous amount of money (even used off of ebay) and Iâve had zero problems since. This thing is rated one of the best for a reason and now that I never have to mess with these again Iâm happy I spent the money.
Unfortunately if you want a portable option, the CalDigit TS3+ does not apply. Itâs solid enough that Iâd consider their other devices though.
Iâve been looking into docs recently, and donât know much about them. Should I be considering Linux compatibility? Do you know if the CalDigit TS3/4 specifically is compatible with Linux?
I can only speak to the the TS3. It works just as well on my Linux Thinkpad as it does with a Mac. I swap back and forth almost daily with no issues. Iâve used a few distros but am settled on Arch.
Iâve been putting off learning Rust macros because they feel too hard. Compare that to Crystal macros for example.
Rustâs declarative macros (â
macro_rulesâ) seem pretty similar. Theyâre a bit more verbose, sure, and a little harder to read maybe. But fundamentally similar in that theyâre declarative.Procedural macros, where you write arbitrary code to produce an AST from an AST, are definitely more difficultâbut also more powerful.
proc macros are token -> token conversions instead of AST -> AST conversions, itâs one of the reasons you need quote/syn for most macro crates. The rfc explains the reasoning here, the big one being so changes to the AST donât break procedural macros.
The core message of the blog post - that you can get away with the worst kind of password if youâre using hardware MFA - is right on the money.
But! Password re-use is what lets people log into all of your (non-MFA) accounts after finding one password in a hacked dump from 2013 that theyâve been chewing on for as long as it takes. Some online vendors probably still arenât salting passwords correctly, making password recovery almost trivial with modern hardware / usual end user password lengths. It wonât show up in these password scan analyses, but can be far more devastating: I still think itâs good advice to not re-use passwords. Write them down in a book, use a password manager, whatever.
& to reiterate that core message: using a 2FA hardware key is the single most effective thing you can do to secure your online accounts. Even if you canât be bothered with 2FA on most of your accounts (which is fair enough) using it on your primary email account will make a huge, huge difference. Anyone with access to the email you used to create other accounts can use that email to reset every account you used it for, so itâs your most important online identity.
The cheap Yubico keys are ÂŁ25 or so. How much will a compromise of your Gmail (Outlook / whatever) cost you?
On one hand, using a Yubikey (or other FIDO device) is extremely easy. Plug it in, touch it. The end. We really just need to expand server-side support.
On the other hand, how do people keep their hardware key always on them? I find this essentially impossible.
You can make it easier by having multiple Yubikeys, including one permanently attached to each device.
Same as the house keys - on a keyring. If thatâs not in my pocket, somethingâs wrong.
Do you carry your keyring around your house/apartment? Maybe my problem is that Iâm unbearably lazy and hate having to get up and walk over to go get my Yubikey from across the room
Itâs in my pocket literally all the time Iâm not in bed or in water. Same with the phone. Iâd keep forgetting them otherwise.
The first thing I do after putting on my trousers is put wallet, phone and keys in my pockets, watch on my wrist and wedding ring on my finger.
If I left my keys out of my pocket, I would be worried about locking myself out of the house accidentally.
If youâre using a YubiKey to log into a web site, the odds are that itâs using WebAuthn these days. You donât need a separate token with WebAuthn, you can use the TPM to protect your credentials and have the OS unlock them with biometrics. Most of the works stuff I use is like this: I tough a fingerprint reader on my machine to log in. I might not have my keys with me, but I definitely do have a computer with me when I want to log in. When I want to log in from another machine, my phone can be used to authorise a new device (either registering a new private key or temporarily) by sending a notification to my phone that requires the same process (touch the fingerprint reader and then approve) to log in.
Mine lives on my keyring, so I always have it on me.
Working on a new markup language for prose (working name: Sona). Example here. Thereâs no parser for this latest version. Still trying to nail down the syntax. Still trying to get rid of the braces. Canât seem to figure out how and still preserve clear hierarchy in a sane way.
The article claims that Rust doesnât have seamless structured concurrency. Rayon (and crossbeamâs scoped threads under the hood) solves most of this problem.
We can use its trait implementations:
Define some expensive functions:
And then pretty quickly max out every core on your computer finding primes.
We can even reference data on the stack. This works because crossbeamâs âscoped threadsâ library enforces that the thread joins back into its parent thread before the reference becomes invalid.
As mentioned in sidenote 10, one can only sometimes reference data on the stack in parent scopes, only things that happen to be Sync. One will often find that their data doesnât implement Sync, because of some trait object buried within. In that situation, one has to refactor to make it Sync somehow.
If one has to refactor existing code, itâs not seamless, IMO.
Something not being
Syncusually means it uses interior mutability without a mutex. Itâs pretty hard to make something magically have a mutex in it; even if you can, itâs a 50/50 chance that itâll be heavily contested and youâll need to restructure it anyway. The other main time Iâve had something not beSyncis when it contains an OS resource or thread-local or such that is explicitly not allowed to be shared between threads, and you generally canât fix that just by find-and-replacing it withMutex<T>.Sometimes complicated things are just complicated, I guess.
See the Rust example linked from the article (I think it was in a side note), something as simple as having a regular trait object somewhere in your data is enough to make something not Sync.
Trait objects can be sync if you write the type as
dyn Trait + Sync.This needs to be written explicitly, because otherwise the trait allows non-sync implementations, and you could have a data race.
Yep, thatâs precisely the issue thatâs in the way of Rustâs seamless concurrency.
Oh, come on. If it doesnât accept buggy code itâs not seamless? Running of single-threaded code on multiple threads should not be âseamlessâ.
Excuse me? I didnât say anything about accepting buggy code.
Youâve complained that non-
Sync(i.e. non-thread-safe) trait objects are prevented from being used in multi-threaded code. If they were allowed, that would be unsound, and âseamlesslyâ allow data race bugs.Just to be clear: Rust trait objects can be compatible with multi-threaded code. You donât need to eliminate them or work around them. They just come in two flavors: thread-safe and non-thread-safe. If you have a non-thread-safe trait object, then of course you canât âseamlesslyâ use it, as that would be incorrect. But when you have
dyn Trait + Send + Sync, it just works in both single-threaded and multi-threaded contexts.Thatâs a pretty big misinterpretation of my words. Iâm happy to clarify what Iâve said.
I am not suggesting any changes to Rust to make safe seamless concurrency possible; Iâm not sure thatâs even possible, without introducing a lot of complexity to the language. Iâm not saying it should change its rules to allow use of non-Sync/non-Send data in multi-threaded code. Iâm not saying anything at all about changing Rust.
Iâm simply stating that, because Rust requires refactoring existing code (for example deeply changing trait objects and other existing data to be
Sync), it cannot do this seamless concurrency. If youâre really curious about the root cause of why Rust requires Sync and Send, and therefore why it canât do seamless concurrency, let me know and I can enlighten. All I mean so far is that Rust cannot do this, because of its own decisions and limitations.But by this definition, languages that care about correctness of multi-threaded code canât be âseamlessâ. Because otherwise you can always be in a situation where youâre dealing with code that either actually isnât thread-safe, or incorrectly declares itself to be thread-unsafe, and you need to fix it instead of having compiler ignore the problem and let you run it anyway. From Rustâs perspective the refactoring youâre describing is a bug fix, because this situation would not happen in properly written code.
Thatâs not true; the article shows an example where a language can safely have multiple threads concurrently reading the same data, without refactoring.
It also links to an example showing that it is not possible to do safely in Rust, without refactoring. Given that it is possible, but not in Rust, it indicates that the problem is with Rust here.
Yes, if we tried this in Rust, it would be a bug in that Rust code, because Rust canât safely express this pattern. That doesnât mean it would be a bug in other languages.
Youâre disqualifying Rust for not being able to safely express this pattern? But you include OpenMP, which doesnât even have these guarantees in the first place? How is that not favoring lack of safety checks as seamlessness?
I see youâre bringing up OpenMP suddenly, when our conversation so far hasnât been comparing C to Rust, it has been about safe languages.
Perhaps our conversationâs context didnât make it clear enough, so Iâll be more specific: Rust cannot support seamless fearless structured concurrency, but the article shows that we can have seamless fearless structured concurrency in another language, such as Vale.
Iâm sure youâre discussing in good faith, and not intentionally straw-manning what Iâm saying, so Iâm happy to keep clarifying. I always enjoy exploring whatâs possible in language design with those who are honest and curious.
Iâm just contesting what can be called âseamlessâ, which is more of argument about semantics of the word, and tangential to the Vale language, so itâs probably not worth digging this further. Thank you for your responses.
Yeah, it would be interesting to have trait objects follow the same rules as deriving traits, same way that new types are automatically
Send+Sync+Dropwhen they can be unless you specify otherwise. ie, sayingdyn Traitusually meansdyn Trait+Send+Sync+Drop. It would probably be a breaking change to do that, so will probably never happen, but I wonder whether itâs one of those things would abruptly spiral out of control and make everything in the world break, or whether it would be a minor inconvenience to implement and then mostly just work everywhere.DNS is full of confusing jargon. At $work we do R&D closely related to DNS and run into this all the time.
Example: what is a âbase domain nameâ? If you go by ICANN, the BDN for âalice.github.ioâ is âioâ, but if you go by the Public Suffix List, the âbase domainâ (more often called a âsiteâ in this context), is âgithub.ioâ because that definition is more relavent for security and privacy on The Web.
A more accurate name that someone was kind enough to invent is TLD+1 (vs eTLD+1) so itâs clear which we mean. Iâve seen âBDNâ refer to both, but more commonly TLD+1.
But somehow, in the midst of all this jargon, we still donât have a reliable way to refer to the â+1â label in TLD+1, even though itâs the most important label in the group. In âapple.comâ, what is âappleâ? At work we call it the âprimary name labelâ, though that phrase also misses the subtlety of distinguishing between TLDs and eTLDs, and isnât widely used as far as I know. (If someone knows official terminology for this, please tell me.)
And then thereâs the hostname/FQDN/domain triplet that all pretty much mean the same thing (until they donât).
And then thereâs the difference between a DNS proxy (sometimes called a forwarding resolver) and a DNS server, which is fairly close to the point made in the article. As far as I can tell, the difference is in whether or not the server caches responses. Or at least thatâs where I think the difference should be. Some people distinguish between proxies and resolvers. Others donât.
Also âlabelâ vs âsubdomainâ. Thatâs a good one. Is the âroot domainâ a subdomain? Itâs usually considered to be an empty label. But if root is a subdomain, shouldnât it be a subdomain of something?
Thereâs more fun in DNS (think about how domains are supposed to be case-insensitive in a Unicode world (also domains arenât necessarily Unicode and can be arbitrary bytes)), but Iâll leave it there before I get too much further off-topic.