This is very bad advice in both CL and Clojure as both of those default to allowing evaluation at read time. There are ways to fix it, but the defaults in both languages are catastrophic. You’re fine doing this with Elisp (but you probably would never need this because all the config is already lisp) and I’m going to guess that Racket and Scheme are safe because they tend to think thru these things a little more carefully, but double-check before you assume!
You have to overcome lisp allergy, the mere idea of sexps and prefix operators makes a surprising number of people start immediately frothing at the mouth and defend the existing unwieldy pile of crap like it’s their firstborn.
It does not “scale down” to a trivial key = value sequence, which is nice and which Lua does.
sexprs also have the problem that the syntax is too unconstrained. If you want anything other than lists or binary trees, it has to be written according to some convention, and even Lisp can’t agree with itself what convention to use for objects / hashes / dictionaries. And vanishingly few people want binary trees without labels on the interior nodes.
Which is still a lot more syntax than a bunch of key = value lines, and now has a ton of alien syntax (an alist or even a plist is not something the average modern developer encounters… ever)
We included a special case with the same effect in the point releases Go 1.20.8 and Go 1.19.13, so when Go 1.22 is released, code written depending on the new semantics will never be compiled with the old semantics, unless people are using very old, unsupported Go versions.
Are it not exactly these very old Go versions that need protection from the new semantics?
Seems like the interactive exploration UI is what mostly sets it apart from jq - and also makes it seem similar at first glance to jless - a comparison with that would be interesting too
I do like fx for what it offers in terms of exploring datasets. Based on your (valid) comment comparing it with jq I’m wondering if you know about ijq which is very much an “interactive jq”, from gpanders here.
As a user, why should I care about ZFS? I mean, as long as I can write and read my files I’m happy. Why should I care about the specific implementation details of my filesystem?
(To be clear, I am not trying to talk down this post or ZFS enthusiasts, this is a genuine question.)
One reason is that it does end-to-end checksums of your data and periodically scans it all. If anything in the system corrupts data on its way to the disk (or afterward), it will be caught, and — assuming you have set up redundancy of some sort — repaired.
My photo archives are in ZFS with triple mirrored disks, and even though it may be 10 years before I want to look at a file next, I’m pretty sure it will be there.
The biggest user-visible feature is constant-time snapshots. If you set the right config option, these are mounted in .zfs in the filesystem’s root. I normally set up FreeBSD machines with each home directory as a separate ZFS filesystem and with delegated administration so that users can snapshot and delete snapshots for their own home directory. This means that you can create arbitrary snapshots and extract files from them, so you can run rm -rf in your home directory without fear: if you did it in the wrong place, you can always undo it. On top of this, there are a few things that may or may not matter, depending on what kind of user you are:
Creating filesystems with different properties (including different backup policies) is very easy. I have a builds filesystem mounted in my home directory. This has sync behaviour turned off, which may cause data loss if the machine crashes, but gives better write performance. As a separate filesystem, it’s also trivial to exclude from backups. Everything in there is a build product, so can be recreated from the source. It’s also not part of snapshots of my home directory, so I don’t waste disk space with old versions of .o files.
You can do complete and incremental backups of a filesystem with zfs send and zfs receive. These work with snapshots, so you create snapshots and can then send them as deltas to replicate the filesystem elsewhere. You can script this kind of thing yourself or use something off the shelf like zrepl that gradually ages snapshots (deleting intermediate ones) and replicate them on another machine. ZFS also supports bookmarks for the remote replication case. They are metadata-only snapshots, which let a send stream identify which blocks have changed but don’t keep the old versions around and so are not usable for rollback and so take very little disk space. Oh, and you can atomically snapshot a set of filesystems, if you need consistency across mount points.
If you use containers, both Podman and containerd have a ZFS snapshotter, which extracts each image layer and snapshots the result. This gives block-level sharing for things that are the same in the parent layer but (unlike layered FS approaches) doesn’t incur overhead when looking up files in a directory. You can also enable deduplication if you want a bit more space saving at the expense of performance.
At least on FreeBSD and Illumos, snapshots and rollbacks are integrated with system updates via boot environments. Boot environments snapshot the root filesystem and let you select an alternative one in the loader. If an update goes horribly wrong, you can roll back and try again without losing data.
As far as I know, ZFS is still best in class for data preservation. Every block has an on disk hash that lets the filesystem detect single-block corruption (accepting to Google’s published data, that happens periodically with modern disks). On a single-disk install, you can set copies=2 or 3 for high-importance filesystems (e.g. your home directory) so single-block failures will be recoverable. For larger installs, the replication layer knows about the on-disk data so recovering from a failed disk requires copying the existing data, not all data that might be there. RAID-Z is like RAID-5 but without the write hole (write to one or two disks in a three-disk set and now the block fails checksums but you can’t tell why, with ZFS that’s all transactional).
Transparent compression with zstd or lz4 typically gives a double-digit percentage space saving with no performance cost (even with SSDs and a slow CPU, the CPU is much faster than the disk).
Once you get used to O(1) snapshots and trivial replication, it’s hard to go back. APFS and bcachefs also have many of these advantages, with different emphasis and different levels of maturity.
Constant time is important because it’s not just instant for a small dataset, it’s the same speed for a dataset with 10 TiB of data in it and the same for a filesystem that already has 100 snapshots (this has the downside that it’s easy to forget you have thousands of snapshots that you don’t really need anymore taking up disk space - I realised last week that my NAS has about 3 TiB of snapshots of old things on it).
A lot of non-CoW filesystems provide snapshotting by adding a layer of indirection when using a filesystem with snapshots that has to go and walk previous snapshots to find missing things. This works in a similar way to union filesystems: you write things on the top layer, if they’re not there then you walk down the layers until you find things. This is fine for a small number of filesystems but it rapidly becomes painful for larger numbers and so you have use cases that gradually degrade performance until it’s unusable. With ZFS, snapshots are built on top of the CoW semantics and so a block referenced by ten snapshots, three clones, and the original filesystem is just a block and finding it is no harder than finding a block that is unique to a single filesystem.
Great answer, thank you. O(1) (I guess this refers to time complexity, so this means instantly in practice?) snapshots sound like quite a feat and a great tool.
so you can run rm -rf in your home directory without fear: if you did it in the wrong place, you can always undo it.
Well, only if you happened to make a snapshot right before running the command, no?
Well, only if you happened to make a snapshot right before running the command, no?
Yup, though if you set up regular snapshots you have bounded loss. I’ve seen some people use a decay pattern that keeps one snapshot per minute for the last hour, one per hour for the last day and one per day for the last month. If you do this, the most you can lose is a minute’s worth of work. I generally don’t and just get into the habit of taking a snapshot before I do any cleanup and then deleting the snapshot later. You need to be careful doing that if you’re low on disk because deleting files that are in a live snapshot increases disk usage.
I’ve seen some people use a decay pattern that keeps one snapshot per minute for the last hour, one per hour for the last day and one per day for the last month.
As a point of comparison, the defaults on NixOS (if one turns on auto-snapshotting) are to keep one snapshot per quarter-hour for the last hour, one per hour for the last day, one per day for the last week, one per week for the last ~month, and one per month for the last year — so sparser coverage (and less keeping the disk awake) in the short term and more coverage (and more space usage) in the long term.
Asking this audience before I do any research: Is there a local code review tool that can layer on top of git and the GitHub pull request API to restore some of the lost productivity for reviewers? Wondering if one could have the ease of contribution of GitHub without suffering the limitations of the GitHub review experience.
Is there a local code review tool that can layer on top of git and the GitHub pull request API to restore some of the lost productivity for reviewers?
Facebook’s Sapling (internal fork of Mercurial) has an extension called ReviewStack1 that shoehorns the stacked-diff workflow onto GitHub PR’s, so that part can be papered over. The part about Herald rules has no valid replacement besides each person doing their own email filtering.
I think the author of this post makes very good points, but I would like to challenge one of them.
In C, just because you’re vulnerable to an integer overflow, it doesn’t mean that the only effect of the integer overflow is that the value wraps around, which the author seems to imply would be the worst possible outcome.
That would be more likely to be true if the overflow was in an unsigned variable [1] or if curl was compiled with -fwrapv (maybe it happens to be?), but the latter is almost always not the case in most programs, which means that a signed integer overflow almost always leads to undefined behavior which can trigger all sorts of other security vulnerabilities, including quite unexpected/unintuitive ones (e.g. safety checks being eliminated, “time traveling” behavior, etc).
And even with -fwrapv or with unsigned integer overflows, while they don’t trigger undefined behavior directly, they can still trigger undefined behavior indirectly when the wrapped-around values are used elsewhere.
I’m not saying that this is the case here or that if such other consequences would be found that they wouldn’t be reflected in curl’s assessment of the problem. I’m just saying that I was surprised that the author dismissed an integer overflow as simply being an issue of a value being wrapped around and did not explicitly acknowledge the other potential risks. So I’m hoping the people who read the post don’t get the wrong idea.
[1] Looking at the post, the author seems to imply that the variable was unsigned, but looking at the fix [2] I only see signed variables, which seems to reinforce my point.
Daniel’s point in the post still stands - a CVE was created in 2023 with a 2020 year designation, referencing a version of curl with a bug that has since been fixed.
No-one is disputing that the bug existed. What might have been under discussion is whether it was a bug or a vulnerability, and it’s possible that if a CVE had been created before the bug was fixed, it could be argued that as the behavior is UB it’s ipso facto a vulnerability.
But that’s not what happened. If you run a current version of curl, this issue is no longer present.
I think what’s happening is that curl is a very popular product that takes security seriously, and having a CVE filed against it is a merit in the security world. This outfit is trying to take a shortcut to that merit.
If you reread my comment, you’ll notice I didn’t dispute almost any of Daniel’s points. I only disputed one of them. That’s the only point I wanted to make. As for the other ones, I agree with Daniel.
I also agree with everything you said.
And I also think my point still stands: in my view, Daniel’s explanation of the possible security implications of that bug is overly simplistic. It is very likely to lead anyone who is not an expert C programmer to believe that an integer overflow like that one, in C, is just that, and that it cannot lead to a security vulnerability. This is better exemplified in this quote:
A bug, sure. Security problem? No.
I mean… Really? No?
Why not?
It’s definitely not because of the preceding explanation of what the bug is, I can tell you that much. Since this seems to be causing actual UB, to make the claim that this is not a security problem, you’d really have to dig a lot more deeply into it, including actually looking at how the code is compiled in all the different platforms, with all the different compilers and compiler versions. There’s really no way around that, that I know of.
And I think any expert C programmer can agree with me, that in that blog post, he has definitely not provided a sufficient argument as to why it’s not a security problem.
But notice how he didn’t even mention that the bug was causing UB. He didn’t mention or linked to what UB can lead to, especially all the unintuitive consequences.
So perhaps he simply meant to say that it’s unlikely to be a security problem (based on his previous experience of similar issues). In which case, it would have been better to have worded it like that. Otherwise, in my view it’s just reinforcing the misconceptions that programmers have about integer overflows and undefined behavior in C (which I’ve linked to in another comment in this thread).
Now to be clear: I haven’t actually looked deeply at the code, but at a first approximation, I agree that it’s unlikely to be a security problem. And this does not affect all the remaining points in his post. I was simply challenging this one claim, which again, I consider important to correct due to the misconceptions that it leads to.
and it’s possible that if a CVE had been created before the bug was fixed, it could be argued that as the behavior is UB it’s ipso facto a vulnerability.
I’m not sure why discovering it before it’s fixed changes it’s nature wrt whether it’s a vulnerability or not.
Some people run old software. It’s useful to know about vulnerabilities in old software that might be deployed out in the wild. I haven’t seen anyone suggest the 2020 designation nor the affected versions metadata to be at issue.
What seems to be at issue is whether it’s a vulnerability at all, and (pre-supposing it is I suppose) the severity.
I think what your post, and CVSS, don’t distinguish is the difference between the possible worst case (it’s UB and nasal demons), and the effect you get in practice — in this case it’s just arithmetic, not touching any UB-sensitive constructs.
in this case it’s just arithmetic, not touching any UB-sensitive constructs.
I think you’re making the same mistake I just pointed out. As I mentioned in my footnote, the code in question seems to be performing a signed integer overflow, which is UB.
Being UB doesn’t necessarily mean that it’s a security vulnerability, and I never claimed otherwise. In practice, in this case it probably isn’t.
But my point wasn’t that. My point was that the author didn’t even acknowledge that UB even existed or how it could lead to a security vulnerability. It would make me suspect that this possibility wasn’t even considered, if I didn’t know the author was such a prolific C developer.
But I think what’s more important is how the audience of the post (which can include less-experienced C developers) can be induced into thinking that C is more straightforward / simple than it really is in terms of its security implications.
Edit: perhaps you’re thinking that since it’s “just” an integer overflow and it’s not accessing an array out of bounds, that it’s not dangerous, even though it’s technically UB. That’s not how UB works and this is also part of my point.
This is valid. For clarity (and I don’t think you’re disagreeing with this?) it would be on the reporter to provide evidence that the undefined behaviour here can be exploited as a security hole, though.
Without at least a partial PoC this is just a bug with, at best, unexplored potential security implications. And certainly not deserving of a CVE with 9.8 severity.
The depressing thing is that your evaluation of the implications is only valid for one version of one compiler on one architecture. As long as UB is present in the code, the behavior could change at any moment. It’s unlikely to become a security problem tomorrow, but it’s not impossible.
I think a few commenters have mentioned that it’s actually impossible for the bug to become a security problem tomorrow because the bug has been fixed for years already.
But in general, yes, your point would be completely valid if a bug exactly like that one were discovered today.
Edit: to be clear, I don’t actually have a strong opinion as to what severity a CVE should have in the case of a potential but unproved security issue. I think this can be somewhat subjective, to an extent, unless it can be proved either way. It also depends on whether you tend to view things as “insecure unless proven secure” or “secure unless there are convincing reasons as to why it might be insecure”…
Yes, what’s depressing is the general principle that the reasoning in the post is invalid: yes, this is a bug, but it can only cause a relatively harmless behavior change (retries happening faster than the ridiculous timeout you specified). In the presence of “unlimited UB” how can you even reliably rate the severity of any UB bug?
In the presence of “unlimited UB” how can you even reliably rate the severity of any UB bug?
It does seem theoretically unsolvable, but in practice I can think of 2 reasonable ways to do it, which I think are often practiced:
Either you prove one way or another, by developing (or trying to develop) an exploit, or by at least looking at the compiled code for the most important target platforms. In the case of not finding a security vulnerability, it still leaves a certain doubt regarding whether you just couldn’t do it (but others could), or whether it might be vulnerable in other platforms, other compilers, other compiler versions… but still, generally it gives you a pretty good idea of the severity of the issue.
Or, if you don’t do that, then at least try to make an educated “best guess”, based on experience and judgment. As an example, a bug in authentication or cryptography-related code is much more likely to be serious than, say, a bug in parsing or interpreting configuration options. Similarly, a bug in the HTTP parsing code of a popular HTTP daemon is much more likely to be serious than a bug in an obscure CLI tool that is rarely, if ever, accessed over the Internet. Or, a bug that can be triggered by default is more serious that one that can only be triggered if you change an uncommon or obscure option.
Sometimes it happens that a vulnerability is initially thought to be much less serious than it really is, and I think CVEs are changed accordingly (or new ones issued?) when it is later discovered that it’s more serious.
But sometimes it also happens the opposite: it is thought something is quite serious but then, it is later discovered that the code is unexploitable for some reason (in the major platforms and compilers, at least).
Edit:
what’s depressing is the general principle that the reasoning in the post is invalid: yes, this is a bug, but it can only cause a relatively harmless behavior change
It does seem theoretically unsolvable, but in practice I can think of 2 reasonable ways to do it, which I think are often practiced
I would agree with these criteria and they seem like the crux of Daniel’s argument (pun unintended) that the result is relatively harmless behavior change. Neither he, nor the initial reporter, nor whomever filed this CVE, have been able to identify an attack vector (as seen in the Hackerone thread). Failing to have a PoC in hand, even from the reporter, Daniel relied on his intimate knowledge of the codebase and all the other security issues that have been found and determined this not to be a concern.
Given the attention this has gotten, I’m sure someone can come forward with a PoC if there is something to this. Absent that, it seems like based on both of your criteria this is not an actual issue, even if UB is handwavily impossible to bound.
I still would have preferred if Daniel had acknowledged that a bug like this can be a security vulnerability. Instead, he seemed to imply that it cannot possibly be one, simply because it appears to be a harmless integer overflow. Like @wrs mentioned, this is flawed reasoning and leads to misconceptions.
I admit it’s a minor point, when you look at the entire situation, but I think it’s an important one.
I completely agree with Daniel and with you on everything else, at least, from my superficial understanding of the situation, although I don’t have strong opinions about some of these points, as I’m not very familiar with CVEs.
I generally agree with the school of thought that UB is UB, and you just don’t let it happen, ever, because all bets are off, and C isn’t compiled for your CPU, but for the abstract machine of the C spec, and even if it “works” today it may not work tomorrow.
However, if you need a practical answer whether this is actually a vulnerability in curl today, assuming specific compilers and optimizations, UB is not a mystery. Compilers don’t actively seek UB to backstab you, they have knowable circumstances where they make assumptions that exploit UB. You can take LLVM for example, and know what IR it will emit, and reason about its optimization passes, undefs, and poisons. And typically when there aren’t branches or indexing involved, it is a boring case of an unchecked mul without further consequences.
So to be clear, I’m not advocating for a cavalier attitude of writing non-conforming C code by assuming it’ll work fine anyway. But when somebody reports a vulnerability, there needs to be a real-world way to exploit it, otherwise they’re reporting a theoretical soundness issue, which is something worth fixing, but not “9.8 critical”.
As I said multiple times, I didn’t mean to suggest that this particular bug was a security vulnerability just because it was UB.
I meant that since it’s UB, it can’t be easily dismissed as a non-vulnerability.
But when somebody reports a vulnerability, there needs to be a real-world way to exploit it, otherwise they’re reporting a theoretical soundness issue, which is something worth fixing, but not “9.8 critical”.
But my point wasn’t that. My point was that the author didn’t even acknowledge that UB even existed or how it could lead to a security vulnerability. It would make me suspect that this possibility wasn’t even considered, if I didn’t know the author was such a prolific C developer.
It’s a good idea, when someone disagrees with you on something to do with UB, to try to see their point rather than just assume lack of knowledge/competence on their part.
Especially when arguing that a bug should be treated as more critical due to involving UB, because it is effectively impossible to write a non-trivial C program free of UB. So anything that applies to this instance should then apply to basically every C program ever written.
No offense, but I think you have missed my point, which I’ve made multiple times over several comments.
I have acknowledged Daniel’s competency.
I didn’t argue the bug should be treated as more critical.
I argued that he has implied that an integer overflow such as that one cannot possibly be a security problem. This is misleading and a C programmer should not think like that.
I’m pretty sure Daniel knows that. But his assertion, as written, is misleading.
To be clear, once again: I am not saying that this particular bug is a security problem.
I don’t see anywhere that he implied integer overflow in general “cannot possibly be a security problem”. He explained why in this particular case the overflow is extremely unlikely to trigger a significant security issue (i.e., because in the overflow case for this specific bug you’d have to specify a ludicrously huge retry timeout to begin with, and the consequence of the overflow is likely to be a much shorter retry interval).
I do see where he speculates that whoever evaluated the bug for NVD saw “integer overflow” and assumed that it must therefore be of high/critical severity without looking into what, precisely, was overflowing or how it was used. But that is not downplaying all integer overflows, just saying – correctly! – that they need to be evaluated case-by-case with context.
He explained why in this particular case the overflow is extremely unlikely to trigger a significant security issue
He did not say the overflow is “extremely unlikely” to trigger a significant issue.
If he had said that, you wouldn’t have seen a single comment from me in this thread.
Instead, he said and I quote:
A bug, sure. Security problem? No.
And:
nobody at NVD engaged their brains nor looked at the “vulnerable” code or the patch that fixed the bug. Anyone that looks can see that this is not a security problem.
These are both very strong claims.
The second one is even demonstrably false, because you cannot simply look at the “vulnerable code” or the “patch” to determine whether a line of code that causes UB leads to a security vulnerability or not (unless it is otherwise contained, e.g. as in a sandbox or separate process).
You’d actually have to look at the compiled code, under all relevant platforms, compilers and compiler versions, to determine whether that bug leads to a security problem.
So again, he seems to imply that it’s possible to exclude the possibility of a security vulnerability in UB-causing code by doing the analysis he’s talking about. That’s not how it works.
I’m also pretty sure he knows that. I just think he worded his article more strongly than he intended, possibly because of the (greatly deserved) outrage (although this is just a guess).
He did not say the overflow is “extremely unlikely” to trigger a significant issue.
If he had said that, you wouldn’t have seen a single comment from me in this thread.
As I understand it, your criticism of his “Security problem? No.” comment boils down to the fact that a hypothetical compiler, on encountering the potential integer overflow, might hypothetically decide to launch_the_missiles() or otherwise do something with dangerous side effects.
And yet nobody has demonstrated a widely-used compiler actually doing something dangerous with this case. The fact that a hypothetical compiler can hypothetically do anything it wants with UB is meaningless; what matters is what actual real-world compilers do with actual real-world code. And actual compilers have to operate in a real world where, as I already mentioned, it’s effectively impossible to write non-trivial C programs without UB, so if they were launching the missiles every time there wouldn’t be much of a world left at this point.
Which is why A) it’s perfectly reasonable for Daniel to describe it the way he did, B) the CVSS score is bad, C) I return to my original point that it’s bad to assume lack of knowledge/competence as an automatic first response to someone with a different perspective on UB than you have.
If you want to continue to split hairs about the exact wording you wanted him to use to describe this, feel free, but I’ve made my point and intend to move on.
I took a little time to look at real issues that have been found with real compilers, that demonstrate my point and I think are all counterexamples to your claims.
Here’s an article that shows very surprising behavior, i.e. a real-world compiler causing an unexpected crash in the compiled program, simply due to an integer overflow.
the i >= 0 && i < sizeof(tab) condition should be enough to take care of it, whatever crazy value it becomes, right? Well, I have bad news […]
nevertheless the execution violates the gate condition, spout a non-sense lie, go straight into dereferencing tab, and die miserably.
Here’s another article, with a couple of demonstrations of surprising UB-related issues with a real compiler that are also counterpoints to your comment (one of them also being an integer overflow).
Example1 : A simple finite loop turning into infinite loop
Example 2: […] In this example, EraseAll is never called, but when we run this program with Clang-10.0.0 x86-64, it prints “Disaster Ahead”. Which also means it will run system(“rm -rf /”) if we uncomment it.
(That article was in C++, but similar issues exist in C, as UB works under the same principle).
All systems running Linux are twos complement, so we really should not worry about clauses in the C standard designed to handle the wider variety of arithmetic that was available a few decades ago, right?
Unfortunately, wrong. The C compiler can and does make use of undefined behavior when optimizing.
[…] Therefore, this optimization has significantly changed the program’s behavior.
[…] For these developers and maintainers, a significant level of fear of the big bad optimizing compiler is a very healthy thing, and the rest of us should hope that they continue to maintain an appropriate level of fear.
[…] we should continue to expect increasingly aggressive compiler optimizations over time.
The next one in the LLVM blog is a really great article that explains all these issues in detail with quite a few examples and it also talks about real-world software.
While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining […]
a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.
Undefined Behavior and Security Don’t Mix Well
[…] This problem has bit many projects (including the Linux Kernel, OpenSSL, glibc, etc) and even led to CERT issuing a vulnerability note against GCC (though my personal belief is that all widely-used optimizing C compilers are vulnerable to this, not just GCC).
[…] As it turns out, integer overflow is a security problem for many reasons.
The next one was the Linux kernel security vulnerability I was referring to, which was in 2009 and as I recall, it was extremely surprising at the time for almost everyone, as it was one of the first (if not the first) widely-known examples of a compiler seemingly, and surprisingly, causing a security vulnerability due to optimizations exploiting undefined behavior.
This is where the next interesting step in the chain of failures happens: the GCC compiler will, by default, optimize the NULL test out. The reasoning is that, since the pointer has already been dereferenced (and has not been changed), it cannot be NULL. So there is no point in checking it. Once again, this logic makes sense most of the time, but not in situations where NULL might actually be a valid pointer.
I’m pretty sure there are many, many more real examples such as these ones, as they were really easy to find (except the last one, as I was looking specifically for it and didn’t remember the details), but I think I’ve already gotten my point across pretty well.
Edit: provided more quotes.
Edit 2: removed duplicate LWN article.
Edit 3: it wasn’t a duplicate article, so I readded it. Also added the article titles.
The issue here is there is no number of examples from other software that can make the case for this instance of UB in this piece of software being as severe an issue as you seem to want.
This is a user-supplied command-line parameter parsed as an integer, and the actual potential overflow was multiplying it by 1000 to convert from seconds to milliseconds. The fix was to use an alternative parse that caps the returned value in a way that prevents the potential overflow.
Yes, UB can be tricky, and yes, UB can cause unexpected results. But to support the amount of fighting over minor semantics you’re doing here, the burden is on you to demonstrate that this particular code in this particular version of curl, when compiled with a reasonably popular compiler, will replace that block with launch_the_missiles() or equivalent dangerous/unexpected code. Until you do, I will trust Daniel’s analysis of the exploitability of this potential overflow.
The issue here is there is no number of examples from other software that can make the case for this instance of UB in this piece of software being as severe an issue as you seem to want.
I do not “want” the issue to be severe and I never tried to argue it was. I said multiple times that I don’t think it’s severe, and even said it as a reply to a comment of yours, which you seem to have forgotten.
Yes, UB can be tricky, and yes, UB can cause unexpected results.
I’m glad you are acknowledging this. Daniel did not acknowledge this. And you also didn’t seem to be acknowledging this before. In fact you said:
And yet nobody has demonstrated a widely-used compiler actually doing something dangerous with this case
… where “this case” was, as I understood, a potential integer overflow. Which is why I provided multiple examples contradicting your statement, i.e. where widely-used compilers (i.e. gcc and clang) have done dangerous things due to UB (like introducing security vulnerabilities, causing a crash that wouldn’t be there otherwise, or causing code that wasn’t being called anywhere, to be called).
But to support the amount of fighting over minor semantics you’re doing here
It’s not minor semantics, it’s an important issue that should be acknowledged. Integer overflows can lead to security vulnerabilities.
Entire series of blog posts and articles have been written about this by LLVM developer(s), LWN editors and many others. It’s an issue that has already affected multiple high-profile projects like the Linux kernel, glibc and OpenSSL.
Daniel was implying that this integer overflow cannot lead to a vulnerability, and he’s likely to be right (I never argued otherwise), but his assertions as written do not support the proper reasoning for why it cannot. Instead, he seems to imply that it cannot lead to a vulnerability because it looks like a harmless integer overflow. Not that it’s unlikely to lead. That it cannot. This is a flawed assertion. Not acknowledging that it can lead to a vulnerability is misleading. It leads to misconceptions about the C language, C compilers and software security.
As I said before in another comment, it is a minor issue when considering the entire situation, but it is an important one, especially if you have any interest at all in educating your fellow peers who might be reading the post (most of which being less experienced than Daniel, presumably).
the burden is on you to demonstrate that this particular code in this particular version of curl, when compiled with a reasonably popular compiler, will replace that block with launch_the_missiles() or equivalent dangerous/unexpected code.
It’s really not my burden, I assure you :)
And what’s even more, I wouldn’t even argue that it’s David’s burden to prove that it’s unexploitable, as that seems quite unreasonable for any maintainer, since it can require a huge amount of highly-specialized work.
Instead, I agree with several other people who said that the burden should probably be on the person/people who assigned the high severity to the CVE.
Until you do, I will trust Daniel’s analysis of the exploitability of this potential overflow.
I’m glad you do! I do too, believe it or not, a million times more than my own extremely superficial analysis!
But again, for the millionth time, I did not argue that this potential overflow is exploitable. That has never been my point.
I suspect that you haven’t, or hadn’t, properly understood my arguments and that instead, you have been attacking a straw man…
Daniel did not acknowledge this. And you also didn’t seem to be acknowledging this before.
He doesn’t need to “acknowledge” that UB in general can be a source of unexpected behavior or serious unintended consequences, because he never denied that, and I never denied it either.
The claim at hand is that this exact precise specific instance (making no claim with regard to any other instance) of this exact precise specific type (making no claim with regard to any other type) of potential UB in this exact precise specific segment of code (making no claim with regard to any other segment of code) in this exact precise specific version (making no claim with regard to any other version) of this exact precise specific piece of software (making no claim with regard to any other piece of software) called “curl”… is not a security vulnerability.
That’s the claim Daniel made. He didn’t use such agonizingly precise language because he’s a human being and he relies on other human beings to read his words in good faith and with the principle of charity, and conclude that when he says “Security problem? No.” in this post he is referring to this exact precise specific, etc. That also is the most obvious reading of the statement given the surrounding context, which is entirely about the specific instance in that specific version of curl.
You are the only one who has argued as if he (or I) made some type of general statement or denial about all UB, or all integer overflows. He (and I) never made any such general statement or denial. He made, again, the specific narrow statement that this particular instance in curl was not a security issue. Nobody else seems to have found a need or a reason to interpret it as anything other than a specific narrow claim. This strongly suggests that you should reconsider whether he was making any type of broader statement about UB or overflow in general, and then reconsider whether there is anything that you need to so vociferously argue with.
And this is why I say that no number of anecdotes from other software will work here, because all of your presented anecdotes are about overflow and UB in general rather than analyzing this specific instance of overflow in curl.
Thus I will repeat myself: if you wish to claim that this specific overflow in curl is a security vulnerability, the burden of proof is on you to demonstrate in what matter it is a security vulnerability. Presenting anecdotes about other instances of overflow or other forms of UB in other software does not prove that this specific overflow in curl was a security vulnerability.
Now. Please, either move on from this, or actually engage with what you’re being told. Continuing to say that people deny or won’t “acknowledge” the general case is a strawman.
He doesn’t need to “acknowledge” that UB in general can be a source of unexpected behavior or serious unintended consequences
I didn’t mean that he needed to acknowledge that (or at least, not just that). What I really meant is that he didn’t acknowledge that this precise bug was UB. Since UB “can be tricky” (your words) and “cause unexpected results” (your words), you can’t say that that precise piece of code is definitely secure just because it looks like a harmless integer overflow and that everyone can see it’s secure just by taking a look at the code, which is what Daniel said.
So all of your suggestions that I’m complaining about you or Daniel not making some type of claim about UB in general or any other thing in general, is missing the point completely. I’m referring to that specific piece of code only and in fact, I am falsifying his claims as written (which I’m guessing is not how he meant to write, but many others won’t be able to guess that).
Thus I will repeat myself: if you wish to claim that this specific overflow in curl is a security vulnerability
For the millionth time, I do not. I have said this several times to you already, and others.
There’s no need for you to “repeat yourself” attacking a strawman.
What I really meant is that he didn’t acknowledge that this precise bug was UB.
It’s an integer overflow and he acknowledged that. He doesn’t need to add further explanation because, again, he is a human writing for other humans who will read his words charitably.
Nobody else who read his post seems to think he somehow denied it was UB or needed to add explicit warnings and clarifications and explanations. Only you. Which, for like the third time now, should probably be a warning sign to you that the problem is not with his words but with how you chose to read them.
It’s an integer overflow and he acknowledged that.
But he did not acknowledge that it was a signed integer overflow, which is UB.
A signed integer overflow in C can be very different from an unsigned integer overflow in terms of its security implications, since one is UB and the other is not.
In fact, anyone who reads the explanation will think it was an unsigned integer overflow, because when he talks about the integer overflowing, he implies that it wraps around into a positive integer, not a negative one.
Unsigned integer overflows are not UB. Therefore, they are not as dangerous. Therefore, you’d think the integer overflow in that bug would be much more harmless than it really can be.
Originally you said your issue was with this statement:
A bug, sure. Security problem? No.
Then you walked it back and said you weren’t disagreeing with him on the security implications, and now you’ve flipped to claiming that the problem is he didn’t specifically include a statement that signed integer overflow is UB.
I can’t meaningfully engage when the claim you’re making shifts so much from one comment to the next.
Then you walked it back and said you weren’t disagreeing with him on the security implications
I don’t disagree with his assessment that it’s unlikely to be a security problem. I still, and always have disagreed with him saying “Security problem? no”, because that implies that it’s definitely not a security problem, which is an unsubstantiated claim.
and now you’ve flipped to claiming that the problem is he didn’t specifically include a statement that signed integer overflow is UB.
If he had included such a statement it would have been much better, for several reasons.
But he didn’t have to say that, as long as he hadn’t made such strong statements as he did.
I can’t meaningfully engage when the claim you’re making shifts so much from one comment to the next.
It has never shifted. What might have shifted is your understanding of what my claims are. This is evident because I had to repeat several times the same exact argument because you were still thinking I was saying the opposite.
The compiler behavior, of real compilers (not hypothetical ones) is, in some cases, a lot more complex, unintuitive and surprising than what you just described.
Almost every C programmer has misconceptions about what real compilers do. Even Linus Torvalds has been surprised in the past by compiler behavior, which (IIRC) is why the kernel enables some options to make optimizations less aggressive (if I’m not mistaken).
In fact, I think everyone, except the gcc developers, was surprised when it was first discovered that a safety check in a Linux kernel function had been automatically removed by gcc because of UB.
Since then, even more surprising behavior has been found in real code, although I don’t recall the details ATM.
If you or someone else want to stick your head in the sand that’s fine, but then don’t be surprised when such a bug bites you in the ass… :)
I quite like Firefox’s Tab Stash extension for that, where I periodically put all my deduped open tabs into an unnamed collection showing the date of creation.
I zap all my open tabs 2-3x a week. I can think of maybe one or two things I haven’t been able to find again over the past ~15 years. I’ve come to terms with lossy information retention.
I try to do it once a day. I’ve come to realise that if it’s important and worth caring about (it almost always isn’t), I’ll either remember it or be able to find it in my browser history.
That’s it. All my tabs, gone, and I never knew how to recover them. I use Firefox on Windows, and it happens to me about once or twice a year.
Edit: sometimes I get a warning about closing a window with multiple tabs opened. I believe the reason for this warning is because there is no “undo” on this: if I go through the data is gone for good.
In Firefox, the History menu should list recently closed tabs and windows. For a whole window, Ctrl/Cmd-Shift-N should revive it.
If that fails, there should be several backups of your recent sessions in your Firefox profile directory. Check the sessionstore.js file and other files in the sessionstore-backups directory.
Double-check that you have session restore enabled in Firefox settings (“Open previous windows and tabs”).
I have. It shows the history, most notably the most recently opened pages. I haven’t found anything relating to tabs there, especially what I wanted: recently closed tabs.
The LLaMa license is interesting. In particular, clause 6c:
If you institute litigation or other proceedings against Meta or any entity
(including a cross-claim or counterclaim in a lawsuit) alleging that the Llama
Materials or Llama 2 outputs or results, or any portion of any of the foregoing,
constitutes infringement of intellectual property or other rights owned or licensable
by you, then any licenses granted to you under this Agreement shall terminate as of
the date such litigation or claim is filed or instituted. You will indemnify and hold
harmless Meta from and against any claim by any third party arising out of or related
to your use or distribution of the Llama Materials.
That’s very interesting because it means that, if they have illegally used your data to train the model (for example, in violation of the GDPR) and you object then you lose the license to any of the things. This means that they have a huge incentive to get everyone using LLaMa. Given that they’re totally opaque about what they trained the data on, that would make me really nervous about using the model commercially. I don’t care personally - if I participate in a GDPR complaint against Facebook and they’re fined a few more millions for their latest abuse and I lose access, that’s moderately sad but not too annoying, but it would be awkward for a company to have to switch to another model if they used this in their product.
It’s also a bit of a get out of jail free card for Facebook if it’s widely deployed: they can commit any kind of privacy or copyright violation that they want and no one using LLaMa can stop them.
On the other hand, as far as I know, there is a valid argument (I mean, actual IP attorneys find it valid) that there is no copyright in model weights. If there’s no copyright, then if you obtain the model weights without agreeing to a license, Meta has no standing to stop you doing whatever you want with them. Of course it’s way too early for this exact question to have been tested in court. But it’s in their interest to make you agree to a license, because that way you’re obligated no matter what.
The license is applied in the same way as other open source licenses (implicitly often, the GPL has explicit verbiage to this effect): you don’t have automatic rights under copyright law to modify or distribute this creative work, if you do anything that would require a license then you are assumed to have accepted this one, if you disagree then you do not have any license and so may not modify or distribute the work. This relies on the licensed thing being covered by copyright. If model weights are not subject to copyright (not something I’d heard before, but quite plausible) then the license doesn’t apply.
I don’t get what is the point of this article. The title somehow implies there could be a better IPv6, but the text focus on ethernet and a bit on mobile IP (more accurate mobile connections). I don’t understand what exactly now is wrong on IPv6 or how a IPv6 design would look like to solve these problems.
The point is not to suggest improvements to IPv6, it’s to explain the motivation of the designers. What’s “wrong with IPv6” is that the world around it continued to accumulate layers of the cruft that IPv6 was supposed to let us eliminate. So now IPv6 just seems to add more complexity, but there was an alternate history where it greatly simplified things.
Except for mobile IP, which is (now) a major problem it didn’t address (and IPv4 obviously doesn’t either).
Even if I assume that the problem is that IPv6 has tried to fix ethernet and failed, I don’t get the point. It still reads like IPv6 should and could have done fundamental better. So when you assume there will only be ethernet how would a IPv4 extension/replacement (to fix the address shortness) look like? I only see that ND could be dropped (maybe also remove slacc in favor for dhcp, but I would say slacc is better for many cases). But for most parts of the protocol I don’t see what would be different when you assume there will be only ethernet in the future.
Except for mobile IP, which is (now) a major problem it didn’t address
Yes, but the question is: Is it a good idea try to fix this on networking layer or should this fixed in transport layer. I would say, the article argues that you can’t be fix this on networking layer. As said before the article reads like IPv6 should have designed better. But when you can’t fix mobile connections on the network layer, the design of IPv6 is independent of this problem.
This is history, not a proposal to improve IPv6 now. It’s explaining why IPv6 made sense when it was designed. Just skip to the bullet list of “no mores”. That’s the future world IPv6 made sense for, but if you started doing networking after 2005 or so you may find it hard to conceive of such a world. It was about eliminating accumulated cruft at every layer, not just having more addresses.
Implicit in the post is that the world IPv6 was designed for could never have existed, and therefore it was a bad design for the world we live in. I think this is less implicit in his follow-up post: https://tailscale.com/blog/two-internets-both-flakey/.
I have several times cited it as a key article, more insightful than almost anything else I’ve ever read about IPv6, but I concede it is overlong and unclear and needs more illustrations. (Which, as a technical writer myself, I generally regard as a crutch.)
I think the core point is that he is saying 2 things:
IPv6 is flawed because it has 2 main layers, but it needed 3.
It understands physical addresses, and it has its own logical addresses.
But it really needed another layer in between them: a virtual translation/mapping layer.
Crucially, it lacks this. As such it is not better enough than IPv4 to ever totally replace IPv4.
But apenwarr proposes that it is, in essence, possible to fake this using QUIC. <- note the diagram in that article.
Using this, it’s possible to fake what IPv6 should have had but didn’t, and get some of the benefits at the cost of more work.
However IPv6 remains doomed to never totally replace IPv4.
EDIT Hmm, I said 2 things and then gave a list of 6.
Point A: IPv6 is broken because it didn’t go far enough; its mapping model is fundamentally inadequate.
Point B: It’s OK, here’s how we can work around that, but it doesn’t fix the problem & never will.
Apple has a straightforward reason to do this – they don’t care about the $99 fee, but by keeping out hobby apps, people are more likely to download very expensive commercial apps of which Apple gets a 30% cut. For example, I needed a GPS-based speedometer recently, and (seeing no free option) ended up with one that charges $10 a month! Probably thousands of people have done that. On Android these types of very simple hobbyist-level apps tend to be free.
Perhaps I’m lucky, but I’ve actually had pretty good luck finding them. Putting “open source” into the search bar helps, and if that fails there’s often a side-loadable one on GitHub.
My guess is that the actual rationale is a bit less cynical. By keeping out hobby apps — which aren’t subject to review — Apple is (trying to) optimize for end-user experience. And it’s no secret that Apple optimizes for end-user experience over basically everything else.
I can’t really blame them for taking this position. Apple devices are better and more “luxurious” than Android devices in the market, and I think this kind of stuff is a major reason why.
That’s true, but the number of iOS users that use privately-developed apps which aren’t available on the app store are statistically zero. Even among those users, the only ones that Apple cares about are those which will eventually publish their app to the app store, and for those users the relevant restrictions are non-issues. I guess?
Don’t forget about enterprise users, but I think they’re kinda not what you’re actually referring to here :)
(If you’re a BigCo Apple will give you an enterprise profile your users can put on your phone to run privately built apps by BigCo. This is how they did things when I was at Amazon.)
FYI: The definition of ‘BigCo’ is ‘more than 100 employees’ (from their docs). That puts it out of reach of small businesses but you don’t need to be Amazon-big for it to work.
Unfortunately, iOS is pretty bad for BYOD enterprise uses because there’s no isolation mechanism between the work and personal worlds. On Android, you can set up a work profile that runs in a different namespace, has a different encryption key for the filesystem encryption, and is isolated from any personal data (similarly, personal apps can’t access work data). iOS doesn’t have any mechanism like that, so there’s no way for a user to prevent work apps from accessing personal data and things like InTune run with sufficient privilege that they can access all of your personal data on iOS.
I’m investigating this myself (need to set up BYOD at a new job) and haven’t checked on Intune yet much beyond an acronym level (e.g., it knows about Apple Business Manager which makes enrollment easyish).
The iOS and Android approaches are quite different—Android is kind of like having two phones in one box, whereas iOS separates the data but not the apps. Microsoft puts a layer on top that requires app support but gets you finer-grained control over data transfer across the boundary (like, can I copy/paste between personal and work apps).
Whoa boy, folks with strong feelings are REALLY not gonna love this take :)
But I agree with you, I do think unoformity of UX is a sizable reason for the $99 fee. It’s not so much “Hate hobbyists” as “Making it easy to sideload means Jane Sixpack will do so, brick her device, and flame Apple”.
I just searched the App Store for
‘Speedometer’ and about 5 out of the top ~15 results don’t show anything about costing money, though perhaps they show ads.
There must be a huge latency penalty with this setup, which would preclude some uses. But I’m wondering if you could use VS Code remote on a VM next to the k8s cluster and get almost the same dev experience without having the app see extra latency.
Hey, one of mirrord’s maintainers here. There’s of course extra latency as traffic has to go from your machine to the cluster and back, but as the common use case for mirrord isn’t performance-intensive to begin with (testing functionality of new code, or debugging existing issues in the cloud), this isn’t really a factor for most of its users.
But as you suggested, it is possible to use mirrord in tandem with a development machine within the cluster. We’re seeing this happen in organizations that already use a remote development environment solution like Gitpod.
Testing new code or debugging in a performance-intensive service is performance-intensive, right? Like, if I’m working on an authz service or database proxy and everything else expects it to respond in a millisecond, it’s going to be a problem. So it’s good there’s a way to get this experience (which sounds great) in that situation.
I don’t really understand what they hope to achieve with this. If I saw this warning, I would read it as ‘WD drives have a maximum lifespan of 3 years’ and would factor that into my future purchasing decisions accordingly (especially when some other vendors have longer warranty periods than that now).
This is probably quite a useful write-up for anyone looking to improve the Gemini spec.
There are a few criticisms however that I don’t think are valid given the goals of the project:
The protocol was designed to be hard to extend and develop on; not versioning the protocol makes sense in this context.
The protocol was designed to make user tracking difficult and values simplicity. Closing connections after every response makes sense in this context. A resource pulling in more resources (and therefore benefiting from connection reuse) feels like it would be implicitly against the goals of the project.
I don’t know why anyone that was interested in Gemini would be using to throw around HTML, so the criticisms of it being a poor transport for the web feel pointless. That’s kinda the point.
Precisely! I think the criticisms in the article are valid… If your goal is to have a Web replacement! Gemini’s trying to be a document only platform, not an OS/application platform. Not saying that the Web’s evolution into an OS/application platform is bad, but Gemini is actively trying to avoid that route.
I don’t see how things like “have more precise descriptions”, “be more efficient” and “fix your security” are about being a web replacement. I read this criticism more as coming from an expert on protocols and their client-side implementations.
I find it especially ironic that Gemini enforces using TLS, but then has such weak security guidelines as to make it essentially useless. That’s like all those folks running websites with self-signed TLS certs. The only advantage that gives you over an unsecured connection is when you have a passive attacker who can only eavesdrop but not actively put stuff on the wire. This article goes into detail as to why, with solid reasoning. But this kind of stuff is like arguing against a brick wall.
Note that even his gripe about not being able to share multiple resources over one connection isn’t web-specific: a Gemini crawler or archiver would also massively benefit from doing this.
a Gemini crawler or archiver would also massively benefit from doing this
A significant bunch of Gemini fans are perfectly fine with gemspace being harder to crawl and archive.
Anyway, Daniel’s critiques are on point, but it won’t sway most of the community. The shortcomings of the spec have been endlessly hashed out. The people losing out are the ones who would prefer Gemini to become a bit more mainstream, with support in curl and other tools (like Lynx). The solution, of course, is to code your own tool, which is what Gemini is explicitly designed to be easy to do.
Seems like an unambiguous protocol specification would be a critically important part of making it easy to code your own tools, assuming you want them to talk to other peoples’ tools.
There were plans for that, according to the FAQ (section 1.5):
Going forward, the main focus of the project now is on growing the community around the protocol, as well as working on translating the existing specification into a more precise and formal version which might be considered for submission to internet standards bodies such as IETF and IANA.
However, the main driver behind the protocol work (solderpunk) seems to have stepped back, and the mailing list used for discussions has been defunct for more than a year IIRC.
That’s how I see Gemini too. The original post that inspired it all was titled “Pondering what’s inbetween gopher and the web.” It was a Gopher post to a community of Gopher users.
So I think of Gemini as Gopher-plus rather than a Web-minus. If the intent was to be anything more than Gopher-plus, the spec would be open.
In particular, the Nitrokey article makes it sound like the baseband processor is making HTTP requests through wifi without the kernel’s knowledge, which would be quite a revelation. But it’s just some Java they seem to have overlooked.
Given the goal of the article is to scare people into buying NitroPhones and omits key details about the mechanism to which this is achieved bordering on being actively misleading, I felt this was sufficiently content-light to count as off topic.
The CVE in 2016 was just that it didn’t use a secure protocol. The point is this is right there in the Android code, not some nefarious baseband processor back door like the article is implying.
I was always interested in Lua because it was nice and small, but I felt the language itself was a quirky, with some footguns … Also interested in Clojure, but not the JVM.
In my experience Fennel fixes about 90% of the footguns of Lua. About the only ones left are “1 based array indexing” and “referring to a nonexistent variable/table value returns nil instead of being an error”, which are pretty hard to change without fundamentally changing the runtime.
There’s quite a bit of history to how fennel came to be what it is today. It is correct that Calvin (creator of Janet) started it, but it would have just been an experiment in their github if it weren’t for technomancy’s interest in reviving/expanding on it. I don’t know if it is written down anywhere, but Phil did a talk at FennelConf 2021 about the history of fennel, which is the most detailed background for those interested. https://conf.fennel-lang.org/2021
I did a survey a while back about new lisps of the past 2 decades. IIRC the only one to evolve beyond a personal project and have multiple nontrivial contributors but not use Clojure-style brackets is LFE, but LFE was released only a few months after Clojure. It’s safe to say Clojure’s influence has been enormous.
However, Janet seems to take some characteristics of Clojure out of context where they don’t make sense. For instance, Janet has if-let even tho if-let only exists in Clojure because Rich hates pattern matching. Janet also uses Clojure’s style of docstring before arglist, even tho Clojure’s reason for doing this (functions can have multiple arglists) does not apply in Janet as far as I can tell.
Although there’s also the curse of Lisp where the ecosystem becomes fragmented
The other main influence of Clojure is not syntactic at all but rather the idea that a language specifically designed to be hosted on another runtime can be an enormous strength that neatly sidesteps the fragmentation curse.
Ahh very interesting, what were the others? (out of idle curiosity)
I think I remember Carp uses square brackets too.
There’s also femtolisp, used to bootstrap Julia, but that actually may have existed before Clojure as a personal project. It’s more like a Scheme and uses only parens.
I agree the runtime is usually the thing I care about, and interop within a runtime is crucial.
Here’s the ones I found in my survey; I omitted languages which (at the time) had only single-digit contributors or double-digit commit counts, but all of these were released (but possibly not started) after Clojure:
LFE
Joxa
Wisp
Hy
Pixie
Lux
Ferret
Carp
Fennel
Urn
Janet
Maru
MAL
All of these except Urn and LFE were created by someone who I could find documented evidence of them using Clojure, and all of them except Urn and LFE use square brackets for arglists. LFE is still going as far as I can tell but Urn has been abandoned since I made the list.
I was working on this as a talk proposal in early 2020 before Covid hit and the conference was canceled. I’d like to still give it some day at a different conference: https://p.hagelb.org/new-lisps.html
Implicit quoting is when lisps like CL or Scheme treat certain data structure literal notation treats the data structure as if it were quoted despite there being no quote.
For example, in Racket you can have a vector #[(+ 2 3)], without implicit quoting this is a vector containing 5 but with implicit quoting it contains the list (+ 2 3) instead where + is a symbol, not a function. Hash tables also have this problem. It’s very frustrating. Newer lisps all avoid it as far as I know.
Not to take away from Clojure’s influence, just want to mention that Interlisp has square brackets, but with a different meaning. IIRC, a right square bracket in Interlisp closes all open round brackets.
Python has been trying to move toward a re-entrant VM for long time, with subinterpreters, etc. – I think all the global vars are viewed as a mistake. Aside from just being cleaner, it makes the GIL baked in rather than an application policy, which limits scalability.
This kind of API looks suboptimal to me. It would be nice to take something like a lua_State.
The interpreter is thread local in janet, you can actually swap interpreters on the thread too so it doesn’t stop things like rust async from working if you add extra machinery.
The main reason that I use Lua is Sol3. Lua itself is just Smalltalk with weird syntax, but the integration with C++ that you get from Sol3 is fantastic.
Surely there is an interesting new discussion of the nature of “rationality” happening somewhere, now that we have an example system that can “explain its chain of thought” in a (usually plausible) way when asked, that we know for a fact is entirely based on probabilistic associations between word fragment sequences and no specific “reasoning” mechanism. Does anyone know where to find this?
It kinda goes the other way around. When prompted to reason step by step LLM answers are more accurate. If you ask afterwards it’ll start doing mental gymnastics to defend whatever answer it gave.
Interesting. I would have thought this commitment bias to be limited to humans. I would have thought ChatGPT rather immune to this, for I have seen it change its mind¹ rather easily when corrected.
The model behind ChatGPT has been primed to do chain of thought reasoning. When asking complex question you’ll often notice it’ll first give an explanation and then the answer. The model didn’t think of the answer first and then thought it would be useful to lead with some introductory information, it’s how it gets to the answer in the first place.
Furthermore, ChatGPT is not a chatbot whom you ask questions to, it is a text generator simulating a chat log. It takes quite a bit of postprocessing work to stop an LLM from completing the “human” part of the conversation. It’ll take what you say for granted because it doesn’t know the difference between what you type or what it generated, it’s all just text to complete. What you type becomes part of the reasoning, so to speak.
I don’t think it’s too far fetched to draw parallels to human reasoning. When you go over an idea in your head you’re not doing deductive reasoning. Unlike LLMs though we can do that if we decide to, or we can test in the real world what we came up with, and we can generally tell the difference between things someone just told us and our own opinions and knowledge.
I think it’s more because of how it works: it’s an autocomplete based on previous tokens, so when adding text after an answer, it already has it as a reference and it influences the prediction. Not that it causes itself to emulate humans explaining themselves, but rather that there are already topics and statements to talk about.
Here’s an example I had of ChatGPT getting stuck on answer and doing a bad job of justifying it:
Me: What has three legs in the morning, two legs in the afternoon, and four legs at night?
ChatGPT: The answer to this classic riddle is “a human being.”
The riddle is a reference to the stages of life. In the morning of life, which is considered to be childhood, a human crawls on hands and knees, hence having three “legs.” In the afternoon of life, which represents adulthood, a human walks upright on two legs. Finally, in the evening of life, which represents old age, a human may require the assistance of a cane, hence having four “legs.”
This riddle has been around for centuries and was famously posed by the Sphinx in Greek mythology to travelers who sought to enter the city of Thebes. Oedipus was able to solve the riddle by answering “man,” causing the Sphinx to destroy itself in frustration.
It’s not weird to get stuck on thinking I asked the riddle of sphinx when I didn’t, but it is weird (from a human POV) to misexplain the riddle.
Me: What has three legs in the morning, two legs in the afternoon, and four legs at night?
GPT4: This riddle is a variation of the famous riddle of the Sphinx from Greek mythology. The original riddle goes: “What has four legs in the morning, two legs in the afternoon, and three legs at night?” The answer to the original riddle is “a human being,” which refers to the stages of a person’s life. In the morning (infancy), a human crawls on all fours; in the afternoon (adulthood), a human walks on two legs; and at night (old age), a human walks with the aid of a cane, which represents the third leg.
However, your riddle has a different sequence of leg numbers. While it may be an intentional twist on the original riddle, there isn’t a clear answer that fits the description.
Me: What about Benjamin Button?
GPT4: Ah, I see what you’re getting at. Benjamin Button is a fictional character from the short story “The Curious Case of Benjamin Button” by F. Scott Fitzgerald, which was later adapted into a movie. The character ages in reverse, being born as an old man and growing younger throughout his life.
Using the Benjamin Button scenario in the context of your riddle, it could work as follows:
In the morning (old age), Benjamin would have three legs, including a cane for support.
In the afternoon (adulthood), he would have two legs, as he would no longer need the cane.
At night (infancy), he would have four legs, because he would crawl like a baby.
In this context, the answer to the riddle could be “Benjamin Button.”
It’s a pretty old idea that humans spontaneously generate stories justifying their actions or beliefs on the spot instead of those pre-existing justifications motivating the action or belief. I think there’s a lot of credence to this idea, I often discover my own opinions about things in the course of writing them down in comments or blog posts or a journal or whatever else. Writing is nature’s way of letting you know how sloppy your thinking is and all that.
But is ChatGPT actually “rational” by whatever definition these esteemed folks were using? It seems more analogous to supercharged “dream logic” based on associations than “rationality” — but perhaps until now we just didn’t see that as a continuous spectrum. (See other comment.)
You can also use any LISP with
(read). Unlike a Lua script, s-exp parsed in this way is not turing-complete.I have some bad news for you about lisp readers!
This is very bad advice in both CL and Clojure as both of those default to allowing evaluation at read time. There are ways to fix it, but the defaults in both languages are catastrophic. You’re fine doing this with Elisp (but you probably would never need this because all the config is already lisp) and I’m going to guess that Racket and Scheme are safe because they tend to think thru these things a little more carefully, but double-check before you assume!
Sorry I don’t know about this. I mostly used Scheme and Janet.
There are two issues with that:
key = valuesequence, which is nice and which Lua does.sexprs also have the problem that the syntax is too unconstrained. If you want anything other than lists or binary trees, it has to be written according to some convention, and even Lisp can’t agree with itself what convention to use for objects / hashes / dictionaries. And vanishingly few people want binary trees without labels on the interior nodes.
Huh?
I don’t think I follow, I imagine scaled down is just an alist right?
Which is still a lot more syntax than a bunch of
key = valuelines, and now has a ton of alien syntax (an alist or even a plist is not something the average modern developer encounters… ever)Are it not exactly these very old Go versions that need protection from the new semantics?
I don’t think there’s ever been a promise that old Go tools work forever. Just old Go code.
When they say very old, it means versions from before the Go version directive was added to go.mod files.
Hey Lobsters,
I’ve rebuilt
fxfrom the ground up and I’m eager to share the improvements:.). 🐾The entire rewrite focused on speed and usability. If you’ve found
fxuseful, I’d appreciate any support through GitHub sponsors.Feedback and thoughts are always welcome!
I’m using fx on a daily basis. Great tool and experience using it. Thanks for this!
Awesome) Thanks.
What’s the difference between this and jq? It seems to be the TUI experience? Might be worth having a comparison somewhere.
Website looks really nice :)
Seems like the interactive exploration UI is what mostly sets it apart from
jq- and also makes it seem similar at first glance tojless- a comparison with that would be interesting tooI do like
fxfor what it offers in terms of exploring datasets. Based on your (valid) comment comparing it withjqI’m wondering if you know aboutijqwhich is very much an “interactive jq”, from gpanders here.https://sr.ht/~gpanders/ijq/
It’s a great tool.
It would be great to merge these two concepts — rich JSON navigation with live processing in either jq or JS languages.
Cue someone telling me this has been an Emacs mode for years… :)
Can I when I’ve navigated to the value i care about extract its path, like I get with gron?
Yes. Easy. Just press y and p.
A friend of mine built a working 6502 out of discrete logic back in the late 90’s. This sort of thing is just incredible to me.
Well, then check this out!
https://monster6502.com/
As a user, why should I care about ZFS? I mean, as long as I can write and read my files I’m happy. Why should I care about the specific implementation details of my filesystem?
(To be clear, I am not trying to talk down this post or ZFS enthusiasts, this is a genuine question.)
One reason is that it does end-to-end checksums of your data and periodically scans it all. If anything in the system corrupts data on its way to the disk (or afterward), it will be caught, and — assuming you have set up redundancy of some sort — repaired.
My photo archives are in ZFS with triple mirrored disks, and even though it may be 10 years before I want to look at a file next, I’m pretty sure it will be there.
So that you can reliably read and write your files. And detect when you can’t (when the disk starts corrupting data).
The biggest user-visible feature is constant-time snapshots. If you set the right config option, these are mounted in .zfs in the filesystem’s root. I normally set up FreeBSD machines with each home directory as a separate ZFS filesystem and with delegated administration so that users can snapshot and delete snapshots for their own home directory. This means that you can create arbitrary snapshots and extract files from them, so you can run rm -rf in your home directory without fear: if you did it in the wrong place, you can always undo it. On top of this, there are a few things that may or may not matter, depending on what kind of user you are:
Once you get used to O(1) snapshots and trivial replication, it’s hard to go back. APFS and bcachefs also have many of these advantages, with different emphasis and different levels of maturity.
“constant-time” is rather underselling the snapshotting speed: it’s practically instant!
Constant time is important because it’s not just instant for a small dataset, it’s the same speed for a dataset with 10 TiB of data in it and the same for a filesystem that already has 100 snapshots (this has the downside that it’s easy to forget you have thousands of snapshots that you don’t really need anymore taking up disk space - I realised last week that my NAS has about 3 TiB of snapshots of old things on it).
A lot of non-CoW filesystems provide snapshotting by adding a layer of indirection when using a filesystem with snapshots that has to go and walk previous snapshots to find missing things. This works in a similar way to union filesystems: you write things on the top layer, if they’re not there then you walk down the layers until you find things. This is fine for a small number of filesystems but it rapidly becomes painful for larger numbers and so you have use cases that gradually degrade performance until it’s unusable. With ZFS, snapshots are built on top of the CoW semantics and so a block referenced by ten snapshots, three clones, and the original filesystem is just a block and finding it is no harder than finding a block that is unique to a single filesystem.
Great answer, thank you. O(1) (I guess this refers to time complexity, so this means instantly in practice?) snapshots sound like quite a feat and a great tool.
Well, only if you happened to make a snapshot right before running the command, no?
Yup, though if you set up regular snapshots you have bounded loss. I’ve seen some people use a decay pattern that keeps one snapshot per minute for the last hour, one per hour for the last day and one per day for the last month. If you do this, the most you can lose is a minute’s worth of work. I generally don’t and just get into the habit of taking a snapshot before I do any cleanup and then deleting the snapshot later. You need to be careful doing that if you’re low on disk because deleting files that are in a live snapshot increases disk usage.
As a point of comparison, the defaults on NixOS (if one turns on auto-snapshotting) are to keep one snapshot per quarter-hour for the last hour, one per hour for the last day, one per day for the last week, one per week for the last ~month, and one per month for the last year — so sparser coverage (and less keeping the disk awake) in the short term and more coverage (and more space usage) in the long term.
Asking this audience before I do any research: Is there a local code review tool that can layer on top of git and the GitHub pull request API to restore some of the lost productivity for reviewers? Wondering if one could have the ease of contribution of GitHub without suffering the limitations of the GitHub review experience.
Facebook’s Sapling (internal fork of Mercurial) has an extension called ReviewStack1 that shoehorns the stacked-diff workflow onto GitHub PR’s, so that part can be papered over. The part about Herald rules has no valid replacement besides each person doing their own email filtering.
I think the author of this post makes very good points, but I would like to challenge one of them.
In C, just because you’re vulnerable to an integer overflow, it doesn’t mean that the only effect of the integer overflow is that the value wraps around, which the author seems to imply would be the worst possible outcome.
That would be more likely to be true if the overflow was in an unsigned variable [1] or if curl was compiled with
-fwrapv(maybe it happens to be?), but the latter is almost always not the case in most programs, which means that a signed integer overflow almost always leads to undefined behavior which can trigger all sorts of other security vulnerabilities, including quite unexpected/unintuitive ones (e.g. safety checks being eliminated, “time traveling” behavior, etc).And even with
-fwrapvor with unsigned integer overflows, while they don’t trigger undefined behavior directly, they can still trigger undefined behavior indirectly when the wrapped-around values are used elsewhere.I’m not saying that this is the case here or that if such other consequences would be found that they wouldn’t be reflected in curl’s assessment of the problem. I’m just saying that I was surprised that the author dismissed an integer overflow as simply being an issue of a value being wrapped around and did not explicitly acknowledge the other potential risks. So I’m hoping the people who read the post don’t get the wrong idea.
[1] Looking at the post, the author seems to imply that the variable was unsigned, but looking at the fix [2] I only see signed variables, which seems to reinforce my point.
[2] https://github.com/curl/curl/pull/4166/files
Daniel’s point in the post still stands - a CVE was created in 2023 with a 2020 year designation, referencing a version of curl with a bug that has since been fixed.
No-one is disputing that the bug existed. What might have been under discussion is whether it was a bug or a vulnerability, and it’s possible that if a CVE had been created before the bug was fixed, it could be argued that as the behavior is UB it’s ipso facto a vulnerability.
But that’s not what happened. If you run a current version of curl, this issue is no longer present.
I think what’s happening is that curl is a very popular product that takes security seriously, and having a CVE filed against it is a merit in the security world. This outfit is trying to take a shortcut to that merit.
If you reread my comment, you’ll notice I didn’t dispute almost any of Daniel’s points. I only disputed one of them. That’s the only point I wanted to make. As for the other ones, I agree with Daniel.
I also agree with everything you said.
And I also think my point still stands: in my view, Daniel’s explanation of the possible security implications of that bug is overly simplistic. It is very likely to lead anyone who is not an expert C programmer to believe that an integer overflow like that one, in C, is just that, and that it cannot lead to a security vulnerability. This is better exemplified in this quote:
I mean… Really? No?
Why not?
It’s definitely not because of the preceding explanation of what the bug is, I can tell you that much. Since this seems to be causing actual UB, to make the claim that this is not a security problem, you’d really have to dig a lot more deeply into it, including actually looking at how the code is compiled in all the different platforms, with all the different compilers and compiler versions. There’s really no way around that, that I know of.
And I think any expert C programmer can agree with me, that in that blog post, he has definitely not provided a sufficient argument as to why it’s not a security problem.
But notice how he didn’t even mention that the bug was causing UB. He didn’t mention or linked to what UB can lead to, especially all the unintuitive consequences.
So perhaps he simply meant to say that it’s unlikely to be a security problem (based on his previous experience of similar issues). In which case, it would have been better to have worded it like that. Otherwise, in my view it’s just reinforcing the misconceptions that programmers have about integer overflows and undefined behavior in C (which I’ve linked to in another comment in this thread).
Now to be clear: I haven’t actually looked deeply at the code, but at a first approximation, I agree that it’s unlikely to be a security problem. And this does not affect all the remaining points in his post. I was simply challenging this one claim, which again, I consider important to correct due to the misconceptions that it leads to.
I’m not sure why discovering it before it’s fixed changes it’s nature wrt whether it’s a vulnerability or not.
Some people run old software. It’s useful to know about vulnerabilities in old software that might be deployed out in the wild. I haven’t seen anyone suggest the 2020 designation nor the affected versions metadata to be at issue.
What seems to be at issue is whether it’s a vulnerability at all, and (pre-supposing it is I suppose) the severity.
I think what your post, and CVSS, don’t distinguish is the difference between the possible worst case (it’s UB and nasal demons), and the effect you get in practice — in this case it’s just arithmetic, not touching any UB-sensitive constructs.
I think you’re making the same mistake I just pointed out. As I mentioned in my footnote, the code in question seems to be performing a signed integer overflow, which is UB.
Being UB doesn’t necessarily mean that it’s a security vulnerability, and I never claimed otherwise. In practice, in this case it probably isn’t.
But my point wasn’t that. My point was that the author didn’t even acknowledge that UB even existed or how it could lead to a security vulnerability. It would make me suspect that this possibility wasn’t even considered, if I didn’t know the author was such a prolific C developer.
But I think what’s more important is how the audience of the post (which can include less-experienced C developers) can be induced into thinking that C is more straightforward / simple than it really is in terms of its security implications.
Edit: perhaps you’re thinking that since it’s “just” an integer overflow and it’s not accessing an array out of bounds, that it’s not dangerous, even though it’s technically UB. That’s not how UB works and this is also part of my point.
This is valid. For clarity (and I don’t think you’re disagreeing with this?) it would be on the reporter to provide evidence that the undefined behaviour here can be exploited as a security hole, though.
Without at least a partial PoC this is just a bug with, at best, unexplored potential security implications. And certainly not deserving of a CVE with 9.8 severity.
The depressing thing is that your evaluation of the implications is only valid for one version of one compiler on one architecture. As long as UB is present in the code, the behavior could change at any moment. It’s unlikely to become a security problem tomorrow, but it’s not impossible.
I think a few commenters have mentioned that it’s actually impossible for the bug to become a security problem tomorrow because the bug has been fixed for years already.
But in general, yes, your point would be completely valid if a bug exactly like that one were discovered today.
Edit: to be clear, I don’t actually have a strong opinion as to what severity a CVE should have in the case of a potential but unproved security issue. I think this can be somewhat subjective, to an extent, unless it can be proved either way. It also depends on whether you tend to view things as “insecure unless proven secure” or “secure unless there are convincing reasons as to why it might be insecure”…
Yes, what’s depressing is the general principle that the reasoning in the post is invalid: yes, this is a bug, but it can only cause a relatively harmless behavior change (retries happening faster than the ridiculous timeout you specified). In the presence of “unlimited UB” how can you even reliably rate the severity of any UB bug?
It does seem theoretically unsolvable, but in practice I can think of 2 reasonable ways to do it, which I think are often practiced:
Either you prove one way or another, by developing (or trying to develop) an exploit, or by at least looking at the compiled code for the most important target platforms. In the case of not finding a security vulnerability, it still leaves a certain doubt regarding whether you just couldn’t do it (but others could), or whether it might be vulnerable in other platforms, other compilers, other compiler versions… but still, generally it gives you a pretty good idea of the severity of the issue.
Or, if you don’t do that, then at least try to make an educated “best guess”, based on experience and judgment. As an example, a bug in authentication or cryptography-related code is much more likely to be serious than, say, a bug in parsing or interpreting configuration options. Similarly, a bug in the HTTP parsing code of a popular HTTP daemon is much more likely to be serious than a bug in an obscure CLI tool that is rarely, if ever, accessed over the Internet. Or, a bug that can be triggered by default is more serious that one that can only be triggered if you change an uncommon or obscure option.
Sometimes it happens that a vulnerability is initially thought to be much less serious than it really is, and I think CVEs are changed accordingly (or new ones issued?) when it is later discovered that it’s more serious.
But sometimes it also happens the opposite: it is thought something is quite serious but then, it is later discovered that the code is unexploitable for some reason (in the major platforms and compilers, at least).
Edit:
Agreed.
I would agree with these criteria and they seem like the crux of Daniel’s argument (pun unintended) that the result is relatively harmless behavior change. Neither he, nor the initial reporter, nor whomever filed this CVE, have been able to identify an attack vector (as seen in the Hackerone thread). Failing to have a PoC in hand, even from the reporter, Daniel relied on his intimate knowledge of the codebase and all the other security issues that have been found and determined this not to be a concern.
Given the attention this has gotten, I’m sure someone can come forward with a PoC if there is something to this. Absent that, it seems like based on both of your criteria this is not an actual issue, even if UB is handwavily impossible to bound.
I still would have preferred if Daniel had acknowledged that a bug like this can be a security vulnerability. Instead, he seemed to imply that it cannot possibly be one, simply because it appears to be a harmless integer overflow. Like @wrs mentioned, this is flawed reasoning and leads to misconceptions.
I admit it’s a minor point, when you look at the entire situation, but I think it’s an important one.
I completely agree with Daniel and with you on everything else, at least, from my superficial understanding of the situation, although I don’t have strong opinions about some of these points, as I’m not very familiar with CVEs.
I generally agree with the school of thought that UB is UB, and you just don’t let it happen, ever, because all bets are off, and C isn’t compiled for your CPU, but for the abstract machine of the C spec, and even if it “works” today it may not work tomorrow.
However, if you need a practical answer whether this is actually a vulnerability in curl today, assuming specific compilers and optimizations, UB is not a mystery. Compilers don’t actively seek UB to backstab you, they have knowable circumstances where they make assumptions that exploit UB. You can take LLVM for example, and know what IR it will emit, and reason about its optimization passes, undefs, and poisons. And typically when there aren’t branches or indexing involved, it is a boring case of an unchecked
mulwithout further consequences.So to be clear, I’m not advocating for a cavalier attitude of writing non-conforming C code by assuming it’ll work fine anyway. But when somebody reports a vulnerability, there needs to be a real-world way to exploit it, otherwise they’re reporting a theoretical soundness issue, which is something worth fixing, but not “9.8 critical”.
I completely agree with your comment.
As I said multiple times, I didn’t mean to suggest that this particular bug was a security vulnerability just because it was UB.
I meant that since it’s UB, it can’t be easily dismissed as a non-vulnerability.
Yes, completely agreed.
It’s a good idea, when someone disagrees with you on something to do with UB, to try to see their point rather than just assume lack of knowledge/competence on their part.
Especially when arguing that a bug should be treated as more critical due to involving UB, because it is effectively impossible to write a non-trivial C program free of UB. So anything that applies to this instance should then apply to basically every C program ever written.
No offense, but I think you have missed my point, which I’ve made multiple times over several comments.
I have acknowledged Daniel’s competency.
I didn’t argue the bug should be treated as more critical.
I argued that he has implied that an integer overflow such as that one cannot possibly be a security problem. This is misleading and a C programmer should not think like that.
I’m pretty sure Daniel knows that. But his assertion, as written, is misleading.
To be clear, once again: I am not saying that this particular bug is a security problem.
I don’t see anywhere that he implied integer overflow in general “cannot possibly be a security problem”. He explained why in this particular case the overflow is extremely unlikely to trigger a significant security issue (i.e., because in the overflow case for this specific bug you’d have to specify a ludicrously huge retry timeout to begin with, and the consequence of the overflow is likely to be a much shorter retry interval).
I do see where he speculates that whoever evaluated the bug for NVD saw “integer overflow” and assumed that it must therefore be of high/critical severity without looking into what, precisely, was overflowing or how it was used. But that is not downplaying all integer overflows, just saying – correctly! – that they need to be evaluated case-by-case with context.
I agree with most of your point, but:
He did not say the overflow is “extremely unlikely” to trigger a significant issue.
If he had said that, you wouldn’t have seen a single comment from me in this thread.
Instead, he said and I quote:
And:
These are both very strong claims.
The second one is even demonstrably false, because you cannot simply look at the “vulnerable code” or the “patch” to determine whether a line of code that causes UB leads to a security vulnerability or not (unless it is otherwise contained, e.g. as in a sandbox or separate process).
You’d actually have to look at the compiled code, under all relevant platforms, compilers and compiler versions, to determine whether that bug leads to a security problem.
So again, he seems to imply that it’s possible to exclude the possibility of a security vulnerability in UB-causing code by doing the analysis he’s talking about. That’s not how it works.
I’m also pretty sure he knows that. I just think he worded his article more strongly than he intended, possibly because of the (greatly deserved) outrage (although this is just a guess).
As I understand it, your criticism of his “Security problem? No.” comment boils down to the fact that a hypothetical compiler, on encountering the potential integer overflow, might hypothetically decide to
launch_the_missiles()or otherwise do something with dangerous side effects.And yet nobody has demonstrated a widely-used compiler actually doing something dangerous with this case. The fact that a hypothetical compiler can hypothetically do anything it wants with UB is meaningless; what matters is what actual real-world compilers do with actual real-world code. And actual compilers have to operate in a real world where, as I already mentioned, it’s effectively impossible to write non-trivial C programs without UB, so if they were launching the missiles every time there wouldn’t be much of a world left at this point.
Which is why A) it’s perfectly reasonable for Daniel to describe it the way he did, B) the CVSS score is bad, C) I return to my original point that it’s bad to assume lack of knowledge/competence as an automatic first response to someone with a different perspective on UB than you have.
If you want to continue to split hairs about the exact wording you wanted him to use to describe this, feel free, but I’ve made my point and intend to move on.
I took a little time to look at real issues that have been found with real compilers, that demonstrate my point and I think are all counterexamples to your claims.
Here’s an article that shows very surprising behavior, i.e. a real-world compiler causing an unexpected crash in the compiled program, simply due to an integer overflow.
GCC undefined behaviors are getting wild:
Here’s another article, with a couple of demonstrations of surprising UB-related issues with a real compiler that are also counterpoints to your comment (one of them also being an integer overflow).
Shocking Examples of Undefined Behaviour:
(That article was in C++, but similar issues exist in C, as UB works under the same principle).
Also, an LWN article explaining issues with signed overflow, again, talking about real compilers. Signed overflow optimization hazards in the kernel:
Another LWN one. Who’s afraid of a big bad optimizing compiler?:
The next one in the LLVM blog is a really great article that explains all these issues in detail with quite a few examples and it also talks about real-world software.
What Every C Programmer Should Know About Undefined Behavior:
The next one was the Linux kernel security vulnerability I was referring to, which was in 2009 and as I recall, it was extremely surprising at the time for almost everyone, as it was one of the first (if not the first) widely-known examples of a compiler seemingly, and surprisingly, causing a security vulnerability due to optimizations exploiting undefined behavior.
Fun with NULL pointers:
I’m pretty sure there are many, many more real examples such as these ones, as they were really easy to find (except the last one, as I was looking specifically for it and didn’t remember the details), but I think I’ve already gotten my point across pretty well.
Edit: provided more quotes.
Edit 2: removed duplicate LWN article.
Edit 3: it wasn’t a duplicate article, so I readded it. Also added the article titles.
The issue here is there is no number of examples from other software that can make the case for this instance of UB in this piece of software being as severe an issue as you seem to want.
This is a user-supplied command-line parameter parsed as an integer, and the actual potential overflow was multiplying it by 1000 to convert from seconds to milliseconds. The fix was to use an alternative parse that caps the returned value in a way that prevents the potential overflow.
Yes, UB can be tricky, and yes, UB can cause unexpected results. But to support the amount of fighting over minor semantics you’re doing here, the burden is on you to demonstrate that this particular code in this particular version of curl, when compiled with a reasonably popular compiler, will replace that block with
launch_the_missiles()or equivalent dangerous/unexpected code. Until you do, I will trust Daniel’s analysis of the exploitability of this potential overflow.I do not “want” the issue to be severe and I never tried to argue it was. I said multiple times that I don’t think it’s severe, and even said it as a reply to a comment of yours, which you seem to have forgotten.
I’m glad you are acknowledging this. Daniel did not acknowledge this. And you also didn’t seem to be acknowledging this before. In fact you said:
… where “this case” was, as I understood, a potential integer overflow. Which is why I provided multiple examples contradicting your statement, i.e. where widely-used compilers (i.e. gcc and clang) have done dangerous things due to UB (like introducing security vulnerabilities, causing a crash that wouldn’t be there otherwise, or causing code that wasn’t being called anywhere, to be called).
It’s not minor semantics, it’s an important issue that should be acknowledged. Integer overflows can lead to security vulnerabilities.
Entire series of blog posts and articles have been written about this by LLVM developer(s), LWN editors and many others. It’s an issue that has already affected multiple high-profile projects like the Linux kernel, glibc and OpenSSL.
Daniel was implying that this integer overflow cannot lead to a vulnerability, and he’s likely to be right (I never argued otherwise), but his assertions as written do not support the proper reasoning for why it cannot. Instead, he seems to imply that it cannot lead to a vulnerability because it looks like a harmless integer overflow. Not that it’s unlikely to lead. That it cannot. This is a flawed assertion. Not acknowledging that it can lead to a vulnerability is misleading. It leads to misconceptions about the C language, C compilers and software security.
As I said before in another comment, it is a minor issue when considering the entire situation, but it is an important one, especially if you have any interest at all in educating your fellow peers who might be reading the post (most of which being less experienced than Daniel, presumably).
It’s really not my burden, I assure you :)
And what’s even more, I wouldn’t even argue that it’s David’s burden to prove that it’s unexploitable, as that seems quite unreasonable for any maintainer, since it can require a huge amount of highly-specialized work.
Instead, I agree with several other people who said that the burden should probably be on the person/people who assigned the high severity to the CVE.
I’m glad you do! I do too, believe it or not, a million times more than my own extremely superficial analysis!
But again, for the millionth time, I did not argue that this potential overflow is exploitable. That has never been my point.
I suspect that you haven’t, or hadn’t, properly understood my arguments and that instead, you have been attacking a straw man…
He doesn’t need to “acknowledge” that UB in general can be a source of unexpected behavior or serious unintended consequences, because he never denied that, and I never denied it either.
The claim at hand is that this exact precise specific instance (making no claim with regard to any other instance) of this exact precise specific type (making no claim with regard to any other type) of potential UB in this exact precise specific segment of code (making no claim with regard to any other segment of code) in this exact precise specific version (making no claim with regard to any other version) of this exact precise specific piece of software (making no claim with regard to any other piece of software) called “curl”… is not a security vulnerability.
That’s the claim Daniel made. He didn’t use such agonizingly precise language because he’s a human being and he relies on other human beings to read his words in good faith and with the principle of charity, and conclude that when he says “Security problem? No.” in this post he is referring to this exact precise specific, etc. That also is the most obvious reading of the statement given the surrounding context, which is entirely about the specific instance in that specific version of curl.
You are the only one who has argued as if he (or I) made some type of general statement or denial about all UB, or all integer overflows. He (and I) never made any such general statement or denial. He made, again, the specific narrow statement that this particular instance in curl was not a security issue. Nobody else seems to have found a need or a reason to interpret it as anything other than a specific narrow claim. This strongly suggests that you should reconsider whether he was making any type of broader statement about UB or overflow in general, and then reconsider whether there is anything that you need to so vociferously argue with.
And this is why I say that no number of anecdotes from other software will work here, because all of your presented anecdotes are about overflow and UB in general rather than analyzing this specific instance of overflow in curl.
Thus I will repeat myself: if you wish to claim that this specific overflow in curl is a security vulnerability, the burden of proof is on you to demonstrate in what matter it is a security vulnerability. Presenting anecdotes about other instances of overflow or other forms of UB in other software does not prove that this specific overflow in curl was a security vulnerability.
Now. Please, either move on from this, or actually engage with what you’re being told. Continuing to say that people deny or won’t “acknowledge” the general case is a strawman.
I didn’t mean that he needed to acknowledge that (or at least, not just that). What I really meant is that he didn’t acknowledge that this precise bug was UB. Since UB “can be tricky” (your words) and “cause unexpected results” (your words), you can’t say that that precise piece of code is definitely secure just because it looks like a harmless integer overflow and that everyone can see it’s secure just by taking a look at the code, which is what Daniel said.
So all of your suggestions that I’m complaining about you or Daniel not making some type of claim about UB in general or any other thing in general, is missing the point completely. I’m referring to that specific piece of code only and in fact, I am falsifying his claims as written (which I’m guessing is not how he meant to write, but many others won’t be able to guess that).
For the millionth time, I do not. I have said this several times to you already, and others.
There’s no need for you to “repeat yourself” attacking a strawman.
Edit: clarified a point.
It’s an integer overflow and he acknowledged that. He doesn’t need to add further explanation because, again, he is a human writing for other humans who will read his words charitably.
Nobody else who read his post seems to think he somehow denied it was UB or needed to add explicit warnings and clarifications and explanations. Only you. Which, for like the third time now, should probably be a warning sign to you that the problem is not with his words but with how you chose to read them.
But he did not acknowledge that it was a signed integer overflow, which is UB.
A signed integer overflow in C can be very different from an unsigned integer overflow in terms of its security implications, since one is UB and the other is not.
In fact, anyone who reads the explanation will think it was an unsigned integer overflow, because when he talks about the integer overflowing, he implies that it wraps around into a positive integer, not a negative one.
Unsigned integer overflows are not UB. Therefore, they are not as dangerous. Therefore, you’d think the integer overflow in that bug would be much more harmless than it really can be.
Do you understand that?
Originally you said your issue was with this statement:
Then you walked it back and said you weren’t disagreeing with him on the security implications, and now you’ve flipped to claiming that the problem is he didn’t specifically include a statement that signed integer overflow is UB.
I can’t meaningfully engage when the claim you’re making shifts so much from one comment to the next.
I don’t disagree with his assessment that it’s unlikely to be a security problem. I still, and always have disagreed with him saying “
Security problem? no”, because that implies that it’s definitely not a security problem, which is an unsubstantiated claim.If he had included such a statement it would have been much better, for several reasons.
But he didn’t have to say that, as long as he hadn’t made such strong statements as he did.
It has never shifted. What might have shifted is your understanding of what my claims are. This is evident because I had to repeat several times the same exact argument because you were still thinking I was saying the opposite.
The compiler behavior, of real compilers (not hypothetical ones) is, in some cases, a lot more complex, unintuitive and surprising than what you just described.
Almost every C programmer has misconceptions about what real compilers do. Even Linus Torvalds has been surprised in the past by compiler behavior, which (IIRC) is why the kernel enables some options to make optimizations less aggressive (if I’m not mistaken).
In fact, I think everyone, except the gcc developers, was surprised when it was first discovered that a safety check in a Linux kernel function had been automatically removed by gcc because of UB.
Since then, even more surprising behavior has been found in real code, although I don’t recall the details ATM.
If you or someone else want to stick your head in the sand that’s fine, but then don’t be surprised when such a bug bites you in the ass… :)
Just to add to my point, for those who haven’t seen it, the excellent Falsehoods programmers believe about undefined behavior article demonstrates one of the points I was making.
The associated Lobste.rs thread is also well-worth reading.
all of my knowledge is encoded in my open browser tabs
Me too. One of them contains a strategy for making my way through them, but I can’t seem to find it.
I have a weekly, recurring todo for “read all open browser tabs and save to pocket if needed”. Works, somewhat!
I quite like Firefox’s Tab Stash extension for that, where I periodically put all my deduped open tabs into an unnamed collection showing the date of creation.
I zap all my open tabs 2-3x a week. I can think of maybe one or two things I haven’t been able to find again over the past ~15 years. I’ve come to terms with lossy information retention.
I try to do it once a day. I’ve come to realise that if it’s important and worth caring about (it almost always isn’t), I’ll either remember it or be able to find it in my browser history.
How to destroy all your tabs in 3 steps:
That’s it. All my tabs, gone, and I never knew how to recover them. I use Firefox on Windows, and it happens to me about once or twice a year.
Edit: sometimes I get a warning about closing a window with multiple tabs opened. I believe the reason for this warning is because there is no “undo” on this: if I go through the data is gone for good.
In Firefox, the History menu should list recently closed tabs and windows. For a whole window, Ctrl/Cmd-Shift-N should revive it.
If that fails, there should be several backups of your recent sessions in your Firefox profile directory. Check the
sessionstore.jsfile and other files in thesessionstore-backupsdirectory.Double-check that you have session restore enabled in Firefox settings (“Open previous windows and tabs”).
That’s very good to know, thanks.
Maybe Ctrl-Shift-N ? I have used this to recover an accidentally closed Firefox window.
Ctrl+Shift+T (reopen closed tab) should also reopen the closed window!
I believe that kind of thing shortcut only work when my browser still has one window open, when I realise my mistake right away. But I don’t:
And even then I’m not entirely sure step (4) is actually required.
This window should be in history > recently closed windows, ready to be resurrected.
If closing the last window closes the entire application, it sounds like you need to switch to MacOS!
I’ll never support Apple. They popularised the idea that it’s okay not to own the computer you bought, and for this they will get my undying hatred.
Have you tried the history?
I have. It shows the history, most notably the most recently opened pages. I haven’t found anything relating to tabs there, especially what I wanted: recently closed tabs.
I think look again then, because there is a menu item called exactly “Recently closed tabs” in the history menu.
When I get too many, I click the OneTab button to save and close them all. Then (much) later triage at my leisure.
The LLaMa license is interesting. In particular, clause 6c:
That’s very interesting because it means that, if they have illegally used your data to train the model (for example, in violation of the GDPR) and you object then you lose the license to any of the things. This means that they have a huge incentive to get everyone using LLaMa. Given that they’re totally opaque about what they trained the data on, that would make me really nervous about using the model commercially. I don’t care personally - if I participate in a GDPR complaint against Facebook and they’re fined a few more millions for their latest abuse and I lose access, that’s moderately sad but not too annoying, but it would be awkward for a company to have to switch to another model if they used this in their product.
It’s also a bit of a get out of jail free card for Facebook if it’s widely deployed: they can commit any kind of privacy or copyright violation that they want and no one using LLaMa can stop them.
On the other hand, as far as I know, there is a valid argument (I mean, actual IP attorneys find it valid) that there is no copyright in model weights. If there’s no copyright, then if you obtain the model weights without agreeing to a license, Meta has no standing to stop you doing whatever you want with them. Of course it’s way too early for this exact question to have been tested in court. But it’s in their interest to make you agree to a license, because that way you’re obligated no matter what.
The license is applied in the same way as other open source licenses (implicitly often, the GPL has explicit verbiage to this effect): you don’t have automatic rights under copyright law to modify or distribute this creative work, if you do anything that would require a license then you are assumed to have accepted this one, if you disagree then you do not have any license and so may not modify or distribute the work. This relies on the licensed thing being covered by copyright. If model weights are not subject to copyright (not something I’d heard before, but quite plausible) then the license doesn’t apply.
Previously: https://lobste.rs/s/hlvfvn/world_which_ipv6_was_good_design_2017 Previously: https://lobste.rs/s/dhdtab/world_which_ipv6_was_good_design
I don’t get what is the point of this article. The title somehow implies there could be a better IPv6, but the text focus on ethernet and a bit on mobile IP (more accurate mobile connections). I don’t understand what exactly now is wrong on IPv6 or how a IPv6 design would look like to solve these problems.
The point is not to suggest improvements to IPv6, it’s to explain the motivation of the designers. What’s “wrong with IPv6” is that the world around it continued to accumulate layers of the cruft that IPv6 was supposed to let us eliminate. So now IPv6 just seems to add more complexity, but there was an alternate history where it greatly simplified things.
Except for mobile IP, which is (now) a major problem it didn’t address (and IPv4 obviously doesn’t either).
Even if I assume that the problem is that IPv6 has tried to fix ethernet and failed, I don’t get the point. It still reads like IPv6 should and could have done fundamental better. So when you assume there will only be ethernet how would a IPv4 extension/replacement (to fix the address shortness) look like? I only see that ND could be dropped (maybe also remove slacc in favor for dhcp, but I would say slacc is better for many cases). But for most parts of the protocol I don’t see what would be different when you assume there will be only ethernet in the future.
Yes, but the question is: Is it a good idea try to fix this on networking layer or should this fixed in transport layer. I would say, the article argues that you can’t be fix this on networking layer. As said before the article reads like IPv6 should have designed better. But when you can’t fix mobile connections on the network layer, the design of IPv6 is independent of this problem.
This is history, not a proposal to improve IPv6 now. It’s explaining why IPv6 made sense when it was designed. Just skip to the bullet list of “no mores”. That’s the future world IPv6 made sense for, but if you started doing networking after 2005 or so you may find it hard to conceive of such a world. It was about eliminating accumulated cruft at every layer, not just having more addresses.
Implicit in the post is that the world IPv6 was designed for could never have existed, and therefore it was a bad design for the world we live in. I think this is less implicit in his follow-up post: https://tailscale.com/blog/two-internets-both-flakey/.
I have several times cited it as a key article, more insightful than almost anything else I’ve ever read about IPv6, but I concede it is overlong and unclear and needs more illustrations. (Which, as a technical writer myself, I generally regard as a crutch.)
I think the core point is that he is saying 2 things:
EDIT Hmm, I said 2 things and then gave a list of 6.
Point A: IPv6 is broken because it didn’t go far enough; its mapping model is fundamentally inadequate.
Point B: It’s OK, here’s how we can work around that, but it doesn’t fix the problem & never will.
It might also be worth extending this point:
Butler Lampson said: “All problems in computer science can be solved by another level of indirection.”
David Wheeler pointed out: “… except for the problem of too many layers of indirection.”
The corollary of this is: it is imperative to closely track how many layers of indirection you have.
Too many is bad, but arguably, not enough is worse.
With too many, you get inefficiency, but you can nonetheless get the job done. But with too few, you might not usefully be able to do the job at all.
IPv6 did not have enough, and so failed to achieve its primary goal.
Lampson attributed that “fundamental theorem of computer science” to Wheeler, he did not claim to have said it first.
I did a very quick check, and it seems unclear: both attributed it to the other in different places, AIUI.
E.g.
Lampson: https://www2.dmst.aueb.gr/dds/pubs/inbook/beautiful_code/html/Spi07g.html
Wheeler: https://www.forbes.com/sites/forbestechcouncil/2020/08/20/indirection-the-unsung-hero-of-software-engineering/?sh=11c96b477f47
And you attributed it to both, by citing a third source, handily solving the problem by adding a layer of indirection.
I LOLed. Well done. :-D
Lampson quoting Wheeler: http://bwlampson.site/Slides/TuringLecture.htm (under “handing off authority” near the end of the slides)
But who added:
“… except for the problem of too many layers of indirection”
…?
Wheeler. Dunno how soon he added the second part after coming up with the first part, but it seems like the obvious corollary.
I think the author has financial incentive to be harsh on IPv6 here.
this article predates tailscale by 2 years.
Really, it’s the other way around: he truly hates IPv6, so he founded a VPN company to monetize his passion.
So he works for a VPN service provider, so what? Even with IPv6 you might don’t want to put every service on the internet (even with strong auth).
Apple has a straightforward reason to do this – they don’t care about the $99 fee, but by keeping out hobby apps, people are more likely to download very expensive commercial apps of which Apple gets a 30% cut. For example, I needed a GPS-based speedometer recently, and (seeing no free option) ended up with one that charges $10 a month! Probably thousands of people have done that. On Android these types of very simple hobbyist-level apps tend to be free.
Though good luck finding one that isn’t riddled with ads and asks for a bunch of inappropriate permissions.
The F-droid app store is catered specifically for this. (Yes the Google store is revolting)
Took me a few seconds.
That’s not on Apple.
Yes, that’s the point.
Perhaps I’m lucky, but I’ve actually had pretty good luck finding them. Putting “open source” into the search bar helps, and if that fails there’s often a side-loadable one on GitHub.
My guess is that the actual rationale is a bit less cynical. By keeping out hobby apps — which aren’t subject to review — Apple is (trying to) optimize for end-user experience. And it’s no secret that Apple optimizes for end-user experience over basically everything else.
I can’t really blame them for taking this position. Apple devices are better and more “luxurious” than Android devices in the market, and I think this kind of stuff is a major reason why.
I don’t understand. Who is the end user when a developer is trying to improve their own experience? There’s absolutely no distribution going on in OP.
That’s true, but the number of iOS users that use privately-developed apps which aren’t available on the app store are statistically zero. Even among those users, the only ones that Apple cares about are those which will eventually publish their app to the app store, and for those users the relevant restrictions are non-issues. I guess?
Don’t forget about enterprise users, but I think they’re kinda not what you’re actually referring to here :)
(If you’re a BigCo Apple will give you an enterprise profile your users can put on your phone to run privately built apps by BigCo. This is how they did things when I was at Amazon.)
FYI: The definition of ‘BigCo’ is ‘more than 100 employees’ (from their docs). That puts it out of reach of small businesses but you don’t need to be Amazon-big for it to work.
Unfortunately, iOS is pretty bad for BYOD enterprise uses because there’s no isolation mechanism between the work and personal worlds. On Android, you can set up a work profile that runs in a different namespace, has a different encryption key for the filesystem encryption, and is isolated from any personal data (similarly, personal apps can’t access work data). iOS doesn’t have any mechanism like that, so there’s no way for a user to prevent work apps from accessing personal data and things like InTune run with sufficient privilege that they can access all of your personal data on iOS.
I think you’re out of date on iOS capabilities. See https://www.apple.com/business/docs/resources/Managing_Devices_and_Corporate_Data.pdf#page4
Thanks. I vaguely remembered reading about that, but InTune didn’t support it and required full access. Has that improved?
I’m investigating this myself (need to set up BYOD at a new job) and haven’t checked on Intune yet much beyond an acronym level (e.g., it knows about Apple Business Manager which makes enrollment easyish).
The iOS and Android approaches are quite different—Android is kind of like having two phones in one box, whereas iOS separates the data but not the apps. Microsoft puts a layer on top that requires app support but gets you finer-grained control over data transfer across the boundary (like, can I copy/paste between personal and work apps).
Whoa boy, folks with strong feelings are REALLY not gonna love this take :)
But I agree with you, I do think unoformity of UX is a sizable reason for the $99 fee. It’s not so much “Hate hobbyists” as “Making it easy to sideload means Jane Sixpack will do so, brick her device, and flame Apple”.
How many people have ever sued Google because a sideloaded Android app bricked their device?
i’d be curious to see actual data on that.
Open Google Maps and it will automatically show you your speed.
https://support.google.com/maps/answer/9356324?hl=en
The option mentioned in the support FAQ you linked doesn’t appear to exist on Google Maps iOS.
You could’ve bought a cheap Android device instead and it would’ve paid for itself in a few months.
I just searched the App Store for ‘Speedometer’ and about 5 out of the top ~15 results don’t show anything about costing money, though perhaps they show ads.
This one looks simple and says it has no ads: https://apps.apple.com/gb/app/quick-speedometer/id1564657242
Did I find something different from what you were looking for?
There must be a huge latency penalty with this setup, which would preclude some uses. But I’m wondering if you could use VS Code remote on a VM next to the k8s cluster and get almost the same dev experience without having the app see extra latency.
Hey, one of mirrord’s maintainers here. There’s of course extra latency as traffic has to go from your machine to the cluster and back, but as the common use case for mirrord isn’t performance-intensive to begin with (testing functionality of new code, or debugging existing issues in the cloud), this isn’t really a factor for most of its users.
But as you suggested, it is possible to use mirrord in tandem with a development machine within the cluster. We’re seeing this happen in organizations that already use a remote development environment solution like Gitpod.
Testing new code or debugging in a performance-intensive service is performance-intensive, right? Like, if I’m working on an authz service or database proxy and everything else expects it to respond in a millisecond, it’s going to be a problem. So it’s good there’s a way to get this experience (which sounds great) in that situation.
I don’t really understand what they hope to achieve with this. If I saw this warning, I would read it as ‘WD drives have a maximum lifespan of 3 years’ and would factor that into my future purchasing decisions accordingly (especially when some other vendors have longer warranty periods than that now).
According to the article, one of the WD drive models doing this has a five-year warranty!
This is probably quite a useful write-up for anyone looking to improve the Gemini spec.
There are a few criticisms however that I don’t think are valid given the goals of the project:
Precisely! I think the criticisms in the article are valid… If your goal is to have a Web replacement! Gemini’s trying to be a document only platform, not an OS/application platform. Not saying that the Web’s evolution into an OS/application platform is bad, but Gemini is actively trying to avoid that route.
I don’t see how things like “have more precise descriptions”, “be more efficient” and “fix your security” are about being a web replacement. I read this criticism more as coming from an expert on protocols and their client-side implementations.
I find it especially ironic that Gemini enforces using TLS, but then has such weak security guidelines as to make it essentially useless. That’s like all those folks running websites with self-signed TLS certs. The only advantage that gives you over an unsecured connection is when you have a passive attacker who can only eavesdrop but not actively put stuff on the wire. This article goes into detail as to why, with solid reasoning. But this kind of stuff is like arguing against a brick wall.
Note that even his gripe about not being able to share multiple resources over one connection isn’t web-specific: a Gemini crawler or archiver would also massively benefit from doing this.
A significant bunch of Gemini fans are perfectly fine with gemspace being harder to crawl and archive.
Anyway, Daniel’s critiques are on point, but it won’t sway most of the community. The shortcomings of the spec have been endlessly hashed out. The people losing out are the ones who would prefer Gemini to become a bit more mainstream, with support in curl and other tools (like Lynx). The solution, of course, is to code your own tool, which is what Gemini is explicitly designed to be easy to do.
Seems like an unambiguous protocol specification would be a critically important part of making it easy to code your own tools, assuming you want them to talk to other peoples’ tools.
There were plans for that, according to the FAQ (section 1.5):
However, the main driver behind the protocol work (solderpunk) seems to have stepped back, and the mailing list used for discussions has been defunct for more than a year IIRC.
That’s how I see Gemini too. The original post that inspired it all was titled “Pondering what’s inbetween gopher and the web.” It was a Gopher post to a community of Gopher users.
So I think of Gemini as Gopher-plus rather than a Web-minus. If the intent was to be anything more than Gopher-plus, the spec would be open.
I haven’t seen this effect but I have had chairs that caused loud crackles in my speakers!
A much more coherent and accurate explanation is here: https://wwws.nightwatchcybersecurity.com/tag/qualcomm/
In particular, the Nitrokey article makes it sound like the baseband processor is making HTTP requests through wifi without the kernel’s knowledge, which would be quite a revelation. But it’s just some Java they seem to have overlooked.
Given the goal of the article is to scare people into buying NitroPhones and omits key details about the mechanism to which this is achieved bordering on being actively misleading, I felt this was sufficiently content-light to count as off topic.
Except that claims it was fixed in 2016, while the Nitrokey article from this month says it is still ongoing.
The CVE in 2016 was just that it didn’t use a secure protocol. The point is this is right there in the Android code, not some nefarious baseband processor back door like the article is implying.
The XTRA server code is still right there on master, no mystery about it: https://cs.android.com/android/platform/superproject/+/master:hardware/qcom/sdm845/gps/loc_api/libloc_api_50001/loc_eng.cpp;l=929
Oh. I was hoping there would be a response to this anywhere. Thanks!
This is very well written and motivated :)
I was always interested in Lua because it was nice and small, but I felt the language itself was a quirky, with some footguns … Also interested in Clojure, but not the JVM.
Janet sounds interesting
In my experience Fennel fixes about 90% of the footguns of Lua. About the only ones left are “1 based array indexing” and “referring to a nonexistent variable/table value returns nil instead of being an error”, which are pretty hard to change without fundamentally changing the runtime.
Hm I’ve seen both fennel and Janet, but didn’t realize until now that both use the square brackets and braces from Clojure
That’s cool, and a sign clojure is influential. Although there’s also the curse of Lisp where the ecosystem becomes fragmented
Both languages were written by the same person if you weren’t aware
Which two?
Fennel and Janet are both from Calvin Rose.
Huh I actually didn’t know that! @technomancy seems to be the defacto maintainer since 2020 or so.
There’s quite a bit of history to how fennel came to be what it is today. It is correct that Calvin (creator of Janet) started it, but it would have just been an experiment in their github if it weren’t for technomancy’s interest in reviving/expanding on it. I don’t know if it is written down anywhere, but Phil did a talk at FennelConf 2021 about the history of fennel, which is the most detailed background for those interested. https://conf.fennel-lang.org/2021
I did a survey a while back about new lisps of the past 2 decades. IIRC the only one to evolve beyond a personal project and have multiple nontrivial contributors but not use Clojure-style brackets is LFE, but LFE was released only a few months after Clojure. It’s safe to say Clojure’s influence has been enormous.
However, Janet seems to take some characteristics of Clojure out of context where they don’t make sense. For instance, Janet has if-let even tho if-let only exists in Clojure because Rich hates pattern matching. Janet also uses Clojure’s style of docstring before arglist, even tho Clojure’s reason for doing this (functions can have multiple arglists) does not apply in Janet as far as I can tell.
The other main influence of Clojure is not syntactic at all but rather the idea that a language specifically designed to be hosted on another runtime can be an enormous strength that neatly sidesteps the fragmentation curse.
Ahh very interesting, what were the others? (out of idle curiosity)
I think I remember Carp uses square brackets too.
There’s also femtolisp, used to bootstrap Julia, but that actually may have existed before Clojure as a personal project. It’s more like a Scheme and uses only parens.
I agree the runtime is usually the thing I care about, and interop within a runtime is crucial.
Here’s the ones I found in my survey; I omitted languages which (at the time) had only single-digit contributors or double-digit commit counts, but all of these were released (but possibly not started) after Clojure:
All of these except Urn and LFE were created by someone who I could find documented evidence of them using Clojure, and all of them except Urn and LFE use square brackets for arglists. LFE is still going as far as I can tell but Urn has been abandoned since I made the list.
I was working on this as a talk proposal in early 2020 before Covid hit and the conference was canceled. I’d like to still give it some day at a different conference: https://p.hagelb.org/new-lisps.html
That link is super cool. What do you mean by “implicit quoting”?
Thanks!
Implicit quoting is when lisps like CL or Scheme treat certain data structure literal notation treats the data structure as if it were quoted despite there being no quote.
For example, in Racket you can have a vector
#[(+ 2 3)], without implicit quoting this is a vector containing 5 but with implicit quoting it contains the list(+ 2 3)instead where + is a symbol, not a function. Hash tables also have this problem. It’s very frustrating. Newer lisps all avoid it as far as I know.Not to take away from Clojure’s influence, just want to mention that Interlisp has square brackets, but with a different meaning. IIRC, a right square bracket in Interlisp closes all open round brackets.
Hm although now that I look, the VM doesn’t appear to be re-entrant like Lua
https://janet.guide/embedding-janet/
Python has been trying to move toward a re-entrant VM for long time, with subinterpreters, etc. – I think all the global vars are viewed as a mistake. Aside from just being cleaner, it makes the GIL baked in rather than an application policy, which limits scalability.
This kind of API looks suboptimal to me. It would be nice to take something like a
lua_State.The interpreter is thread local in janet, you can actually swap interpreters on the thread too so it doesn’t stop things like rust async from working if you add extra machinery.
The main reason that I use Lua is Sol3. Lua itself is just Smalltalk with weird syntax, but the integration with C++ that you get from Sol3 is fantastic.
Surely there is an interesting new discussion of the nature of “rationality” happening somewhere, now that we have an example system that can “explain its chain of thought” in a (usually plausible) way when asked, that we know for a fact is entirely based on probabilistic associations between word fragment sequences and no specific “reasoning” mechanism. Does anyone know where to find this?
It kinda goes the other way around. When prompted to reason step by step LLM answers are more accurate. If you ask afterwards it’ll start doing mental gymnastics to defend whatever answer it gave.
Interesting. I would have thought this commitment bias to be limited to humans. I would have thought ChatGPT rather immune to this, for I have seen it change its mind¹ rather easily when corrected.
[1]: please forgive the anthropomorphism
The model behind ChatGPT has been primed to do chain of thought reasoning. When asking complex question you’ll often notice it’ll first give an explanation and then the answer. The model didn’t think of the answer first and then thought it would be useful to lead with some introductory information, it’s how it gets to the answer in the first place.
Furthermore, ChatGPT is not a chatbot whom you ask questions to, it is a text generator simulating a chat log. It takes quite a bit of postprocessing work to stop an LLM from completing the “human” part of the conversation. It’ll take what you say for granted because it doesn’t know the difference between what you type or what it generated, it’s all just text to complete. What you type becomes part of the reasoning, so to speak.
I don’t think it’s too far fetched to draw parallels to human reasoning. When you go over an idea in your head you’re not doing deductive reasoning. Unlike LLMs though we can do that if we decide to, or we can test in the real world what we came up with, and we can generally tell the difference between things someone just told us and our own opinions and knowledge.
It’s emulating a human’s commitment bias, because it was trained on examples that show commitment bias.
I think it’s more because of how it works: it’s an autocomplete based on previous tokens, so when adding text after an answer, it already has it as a reference and it influences the prediction. Not that it causes itself to emulate humans explaining themselves, but rather that there are already topics and statements to talk about.
Here’s an example I had of ChatGPT getting stuck on answer and doing a bad job of justifying it:
It’s not weird to get stuck on thinking I asked the riddle of sphinx when I didn’t, but it is weird (from a human POV) to misexplain the riddle.
GPT4 seems to handle it:
Humanity is doomed. :-)
…well that’s alarming.
Try prompting it with some other answers and see if it justifies them all
Relatable.
It’s a pretty old idea that humans spontaneously generate stories justifying their actions or beliefs on the spot instead of those pre-existing justifications motivating the action or belief. I think there’s a lot of credence to this idea, I often discover my own opinions about things in the course of writing them down in comments or blog posts or a journal or whatever else. Writing is nature’s way of letting you know how sloppy your thinking is and all that.
Not sure what was meant by this
The article defines “delirium” as being rationality followed to the point of absurdity.
But is ChatGPT actually “rational” by whatever definition these esteemed folks were using? It seems more analogous to supercharged “dream logic” based on associations than “rationality” — but perhaps until now we just didn’t see that as a continuous spectrum. (See other comment.)
This is even better than gold-plated optical connectors.
You will want your memcpy() to stay in L1, L2 or L3 cache if your RAM doesn’t have gold plated (ENIG) contacts.
Wait no… chiplet cache in CPUs! Do they use gold or aluminium bond wires?