If anyone else is planning to create new software that relies on NFSv3: please don’t. 32-bit timestamps are part of the protocol which will run out in the year 2106. There are people alive today who will have to clean up that mess.
This reminds me of when I was a child learning BASIC. I drew a circle to the screen and thought “so this is it. A perfect circle. Wow. It’s a little blockier than I thought, but it’s on a computer, so it must be perfect, right?”
I distinctly remember the moment, when I was 12, that I realised that the opposite of such things was true.
I wrote a BASIC program to repeatedly take the square root of a number, and graph it. Eventually, that result became zero, as opposed to doing what I expected it to do - approach zero. I was deeply confused at the time.
Edited: I was surprised to find as I aged that some of my most vivid memories of childhood are confusion and discovery. And how many of them relate to computers :)
For me, it was logo. We were taught to write programs to draw equilateral triangles and squares and told to draw regular polygons with 5, 6, and 7 sides. After a few tries (I was 7 at the time and had only been programming for a few hours), I wrote a program that generated an n-sides regular polygon. I tried it with 500 and it (very slowly!) drew a circle. I knew about pixels, so I knew it wasn’t really a circle, but realised that a circle was an infinity-sided regular polygon, which felt like a major revelation. It wasn’t until I was about double that age that I learned the maths that explains why that’s an interesting conclusion.
A computer can certainly represent a perfect circle (though it cannot display it). It can also represent the square root of a number, taken as many times as you would like.
On the flip side of the coin, we can formalise what computers do when they do approximate; the computer is then a perfect or near-perfect rendition of these formalisations.
I was upset when I took a summer class that taught us C and the instructor told us about bit precision. It’s a computer, what do you mean it can’t just store all the bits in 0.1!
Yeah. As a kid, it took me a while to understand the difference between the discrete world of digital machines and the continuous world of some mathematical concepts. I heard the term “discretization” a lot but I only truly grasped it after I had more programming experience. The instant feedback of modern programming really accelerates learning.
The slowdown has nothing to do with iterators. And it has nothing to do with Rust. At the core, it’s a very basic and fundamental concurrency bug. As he mentioned in the post, if the programmer refactored the two for loops into a single one instead of an iterator, the exact same thing would have happened. Conversely, if he used two separate iterations, the bug wouldn’t have happened. And of course the situation would be exactly the same in C++ or any other language like that. This is bordering on clickbait, really nothing to see here. I expected a post addressing real performance costs of iterators.
You’re assuming bad faith for what I read as an example how someone would naively (intuitively?) convert a to b, and it’s in Rust and it uses iterators - that’s what happened. Of course this can happen elsewhere, still nothing in this post is wrong.
But yeah, one could probably frame it a little bit different,
Yes, it does. The post clearly states that this is a problem for beginners, and I think you’re just to knowledgeable to even understand the issue. Of course this needs two loops, but the problem is that a beginner might not understand that .map and .for_each get turned into a single loop. In a language like js or ruby, stuff like this would run as two loops. That’s the entire misunderstanding.
This isn’t clickbait, but you aren’t the target audience.
Many years ago, I was at a talk by Robin Milner where he argued that artificial intelligence is the wrong goal and that we should focus on augmented intelligence. Making computers do things that humans can do is not especially useful, making computers and humans able to do things that neither can do in isolation is hugely beneficial. Things like Copilot fit well into that lens, they don’t replace the human they let the human accomplish more.
To the notes at the end, I don’t find it worrying that an LLM can reproduce college essays, at least in the sense that’s implied. I see this as evidence that these forms of assessment are not measuring the right thing. I hope LLMs will be a disaster for the kind of assessment that doesn’t require original thinking yet purports to measure that faculty.
I don’t find it worrying that an LLM can reproduce college essays, at least in the sense that’s implied. I see this as evidence that these forms of assessment are not measuring the right thing.
I agree. When these LLMs reach such high scores on standardized school/university tests, the shocking thing is not how far computers advanced, but how primitive our assessment of skill truly is. That it can be fooled by what amounts to a probabilistic mechanism that generates likely chains of tokens. That is not the kind of skill and resourcefulness you look for when you’re hiring people. True resourcefulness is navigating uncharted waters, not doing what is familiar but doing what is logical, given the circumstances.
It’s tricky. When you design an assessment there are four key things you need to think about:
Validity. Are we actually measuring the right thing? In the simplest case, are we actually measuring something: internal consistency metrics leisure this.
Reliability. Are our results reproducible? If the two cohorts of equal ability sit the same test, will we see the same distribution of results?
Impact. Does the thing that we’re measuring correlate with the thing that people looking at the test score care about? For example, if the test is a qualification used for getting a job or getting into university, does the test score correlate with your ability to do well after being accepted?
Practicality. Can we actually run this test at the scale necessary? Having an expert follow the candidate around for a few weeks while they demonstrate the skill works really well, but tends not to scale (especially if you want reliability, so have to calibrate your experts’ judgement).
A lot of university exams don’t even start with a construct against which they can measure validity. They have some aggregate hand waving towards impact (do people who graduate our course get good jobs and do well in them?) but often the sample sizes are too low. They do care about reliability because that’s the kind of thing that they need to demonstrate for accreditation, but only very loosely. They care a lot about practicality because the people marking assessments are not doing that full time, they’re also doing research.
Even with those caveats, a number of the things where LLMs do well are the kind of assessments that are intended to drive learning rather than give a final score. They are intended to tell the student what they need to study more. Cheating at these is like cheating at exercise.
Many years ago, I was at a talk by Robin Milner where he argued that artificial intelligence is the wrong goal and that we should focus on augmented intelligence. Making computers do things that humans can do is not especially useful, making computers and humans able to do things that neither can do in isolation is hugely beneficial. Things like Copilot fit well into that lens, they don’t replace the human they let the human accomplish more.
I wasn’t at the talk but I think I saw either a recording of it, or he made that point in a later talk that I saw, and it was a pretty defining moment for my outlook on programming, and on technology in general. Based on that, and on my personal observations, I also think rumours of programming being on its way to the urn because of “AI” (by which commenters usually mean LLM) are greatly exaggerated. This article is probably a good example: it has a few good ideas, but the examples are tone-deaf.
I don’t remember where I read it first (I think Norvig quotes it in his big book on the topic, but I mostly skimmed it – my interest in AI is mostly confined to writing things that allow people who are better at math than I am write AI tools) but at some point, someone noted that mankind finally achieved its ancient dream of flying when it stopped trying to emulate the flight of birds, and tried to make things that fly starting from the principles of flight but without attempting to emulate the form of flight.
The examples in this article miss that: they get hung up on code monkey “tasks” (which may ultimately be fixed by incremental progresses in LLMs; ChatGPT 4 fares better on a lot of things, who knows what ChatGPT 6 will be able to do) or pointless programming puzzles. With any luck, something good may come out of that, like the death of interview programming puzzles, but that’s IMHO that’s not where the potential of LLMs lays.
I’m (slowly – the web is really not my platform) writing a tool that uses LLMs for very practical problems. tl;dr wife runs a marketing agency, I’m writing a tool that assists with the analysis of various types of promotional text. It’s very much an analysis tool; it has an interface that vaguely alludes to virtual assistants but the interaction is not chat-driven at all.
After hacking at it for a couple of evenings I’m fully convinced that LLMs will not spell the end of programming, and I’m really enthusiastic about what they can do:
Getting that kind of analysis from a language model would’ve been absolutely impossible 10 years ago, and would’ve required concerted effort from several graduate students to come up with a very narrow model. Getting a very useful analysis (I’m talking things like identifying elements of marketing/sales strategy) is something I’ve been able to do with an investment of like thirty minutes, ten of which were spent yelling at Docker.
Turning that into a useful program absolutely requires programming effort. This isn’t something you can do with a no-code tool. You can get some of the output the program produces with nothing but the ChatGPT interface but it’s extremely unwieldy. Properly organising it and correlating various bits and pieces of output is not something a LLM can do, and being able to do it efficiently and repeatably – which is definitely done by old-fashioned programming – is a big deal.
The examples in the article are relevant in one way – I’ve found LLMs to be remarkably good at doing things that otherwise require tedious and somewhat “machine-like” human effort. For example, despite extracting and correlating a bunch of data from freeform text, and then showing it in standard format, my code has basically no serialization/deserialization logic. If I need ideas extracted from a text, and I need to hand them off in JSON format to the UI layer, I just request them in JSON format. If I need to show formatted text, I just request it in Markdown format. It’s not foolproof: chatgpt-3.5 occasionally hands me invalid JSON or tries to be nice and say “Here’s your JSON first”, so there’s some basic sanitizing logic in there; and there are security implications to it that I haven’t fully come to terms with. However, these look like problems that are perfectly tractable between incremental improvements in LLM (or, for that matter, some format awareness) and some real-life experience.
This part of writing an application may go the way of the Dodo – all that code that takes these two JSON objects and produces this other JSON object in a slightly different format, or takes this CSV document uploaded by the customer and produces this standard BOM table. But bloody hell, good riddance, if anyone dreads a future in which we’ll have to do less CRUD and object mapping, they’re suffering from Stockholm syndrome.
I think things could head in this direction? LLM-generated code is probably difficult to trust. There is some interesting work on synthesis based on contracts / advanced type systems using SAT/SMT. I expect both approaches might converge. We may end up designing programs, with AI helping to fill in the gaps.
Some things are. This is explicitly the goal of copilot. I am not convinced that it’s going in the right direction though because it’s acting as a glorified autocomplete and if typing is the bottleneck when programming then I’m doing something wrong. The kinds of things that I would love to see from LLMs are:
Give me LLM-generated code review comments, trained on code review comments fine tuned with my repo, before I push. Even if these have a moderately high false positive rate, if they can flag things that I should double check them they may be useful.
Natural-language-driven refactoring tools. There are a bunch of mechanical refactoring that I do that are easy to say but hard to automate.
Naming style checking. Clang format can do things like ‘this must be camel case’, but not things like ‘interfaces must be adjectives, classes that implement these interfaces should be noun phrases incorporating the interface’. That kind of thing would be fantastic for automated tooling to do since it’s one of the most time consuming things to get right in good APIs.
Automated flagging of APIs that don’t match (or, worse, almost match) design patterns used elsewhere in the library. This requires fuzzy pattern matching, the kind of thing that LLMs are really good at, but also requires quite a large context window. Humans are often bad at spotting this because their equivalent of a context window is smaller and doesn’t find the places where the library has exposed an interface that’s confusingly similar. I’m the simplest case, this can just flag the same kinds of parameters in different orders but there’s a lot more that it could do.
The main thing here is that they’re all low stakes: it won’t break the code if the LLM is wrong (except the refactorings, but that can be coupled with testing) but it will make the output much nicer for the same human investment.
assuming that the human brain is not the maximum possible intelligence a mind can have, and that artificial intelligences can surpass our intelligence by orders of magnitude, and that they can be run on faster processors with larger data stores, that AIs could be reproduced with a simple ctrl-v ctrl-p– the question should not be “what is AI good for?”, but “extrapolating current trends, how much longer will humans be useful for anything at all?”
assuming that the human brain is not the maximum possible intelligence a mind can have,
That is probably true, though it may be that it devolves into discussions about forms of intelligence. The human brain is an interesting balance of compute vs I/O and it may be that increasing compute power alone is not useful, increasing compute and I/O gives something that is unrecognisable. Given that we currently have a sample size of one for building models of kinds of intelligence, there’s a lot of margin for error in anything that we build.
and that artificial intelligences can surpass our intelligence by orders of magnitude,
That’s a nice hypothetical but no one knows how to build one that even reaches the level of a human yet. The article shows problems that young children can solve but that state of the art LLMs can’t. To exceed by orders of magnitude, they’d first have to reach parity.
That said, there are some things that computers can already do orders of magnitude faster than humans. They can compute the sum of a list of integers many orders of magnitude faster than humans, for example. A human with a spreadsheet can achieve far more than a human without a computer, but the spreadsheet by itself can achieve nothing without human agency framing the problems. There are a lot of places where human+computer could be vastly more useful now than they are, but are being overlooked in the rush to make computer-without-human systems.
and that they can be run on faster processors with larger data stores
Again, that’s also not clear. Human brains are probably not quantum computers, but they are highly connected neural networks. Neurons in the brain can connect to several thousand other neurons. Transistors on silicon chips can connect to single digits of other transistors. Simulating this with matrix multiplication is not space efficient and slows down as the connectivity grows (especially when you use back propagation to represent feed-backwards neural networks). Adding larger data stores increases latency.
So, your argument starts with three axioms, none of which is proven and at least one of which may not be true in the sort to medium term. Logical arguments that start from axioms that are not generally helpful.
extrapolating current trends, how much longer will humans be useful for anything at all?
Extrapolating from current trends, forever. That may not be good extrapolation because this kind of development often happens in bursts. Extrapolating from the difference engine, humans would be useful for menial calculation for a long time, but extrapolating from the first electronic computers gave a clear end point for that utility. AI systems running on quantum computers may be able to express forms of conscious intelligence that are so far beyond humans that they’re unrecognisable, given some clever bit of mathematics that no one has thought of yet.
I think that you’ve begged the setup. What is intelligence? Or, at least, what does it mean to compare intelligences? Is there a lattice of intelligence?
It uses JetBrain’s own analyzer implemented in Kotlin. I have been thinking whether its best for JetBrains to do their own thing, or to use an existing project (racer -> rls -> rust-analyzer) since 2015 and still don’t know the answer! For ecosystem as a whole, diversity is clearly good.
Why would a community founded on open source “love” closed source software to the detriment/obsolescence of that open source software JB provided previously?
They chose licenses that were designed to allow proprietary forks. The MIT license was written for the X Windows system that was intended to be (and was) forked by proprietary UNIX vendors to build a compatible ecosystem of proprietary tools. I presume the Rust folks are excited to see this happening in their ecosystem. If they’d wanted a different kind of ecosystem they’d have chosen a different license.
The software in question is the official system used by Dutch municipalities for tallying the total number of votes after collecting them from individual voting stations. This used to be done with some sort of Excel template, and this has been in the news before for being a pretty iffy way of going about things, after which this particular software was created. So you could say this was quite a serious leak. Elections are coming up in November, so the timing of this is pretty good too.
According to the NOS (a Dutch news organisation), security audits had been done, but the audits did not include the installer. Future audits will include it. This points out the flaw of trusting too much on audits, and not taking a wider picture approach where the entire software development process is audited.
I’m quite surprised that there is an official system for doing this. In a paper election, each candidate (or their party) can provide monitors who watch the count. I’d expect an open electronic system to use multiple, independently developed, systems for this and require that they agree within the margin of error for an election to be certified.
The counting of votes is done at the individual voting stations, and the general public has the right to witness this counting. This system is used for adding up those totals from the stations so that they can be reported “up the chain”, which AFAIK doesn’t happen in public view.
So if I want to attack a Dutch election, this looks like the best place to do it. Having two independent implementations of the aggregation would help a lot there because I’d have to compromise both. Allowing parties to provide their own aggregation software would also help, because then if they disagree with the official tallies you can introduce an audit.
Electronic vote counting seems like such a dangerous and irresponsible thing to do, especially for countries that do not produce their own silicon. I can’t believe that it’s such a widespread practice, given the obvious dangers that are well-known for decades already. It’s so much harder to fake entire paper trails at scale.
Attestation means you can only use approved clients, which is terrible for competition and innovation (sorry, it’s now impossible to make a new browser or OS!)
Sounds really bad. This will make it hard to run non-mainstream platforms.
I guess regulators might step in and demand alternative modes of attestation, like they have done in case of iOS and the single-store monopoly. However, they are exceedingly slow at doing so. For instance, now it is impossible to do online banking in some places if you don’t run iOS or Android. Something as innocent as AOSP is not supported, even if you run the same banking app, due to the lack of SafetyNet attestation.
It’s also not clear to me that policymakers have any particular reason to care about non-commercial platforms or software modification as civil liberties issues. So we might not get the regulation we want…
Honestly not a bad time to start thinking about what the rules OUGHT to be in an ideal world. Even if we can’t get there anytime soon, knowing where we want to go has to help somehow.
The EU has a good reason to care. All of the big OS vendors are based in the US. If things require an attestation that only a US company can provide then this makes it impossible for any EU startup to ever grow to being a big company in this space.
This is especially true when you consider that things like the YouTube and Android are owned by the same US entity. If Google decides to require attestation to watch YouTube videos (so that they know that you aren’t using a client that skips ads, for example) then there is a fairly clear argument under existing monopoly law that they’re using their effective monopoly in one market to ensure that there can be no competition entering another market where they’re a significant player.
I think we need to learn a lot from the problems with DRM here. We pushed back on the technologies, rather than the uses. We didn’t block the technology in standards bodies (in part because there are some non-evil uses, such as access control within an organisation) and we didn’t get regulations on acceptable use. I’d love to see using DRM that prevents users from exercising fair use rights treated as vigilanteism and grounds for having copyright on the work in question revoked. Similarly, anyone using this kind of attestation to lock in clients to a particular platform should be subject to serious antitrust investigation.
The political leaders of the EU have a good reason to care that European keys find their ways into the TPMs of digital devices. They have no reason whatsoever to care about civil liberties of Europeans, or that people can run their own libre software on their devices. Quite the contrary - free computing is a threat to governing bodies, especially those that like to grow in size and power. The TPM menace will only grow worse as time goes on.
My immediate vision is of a future where I end up having to literally drive to the bank to pay my bills because, eventually, I will no longer run a rights-infringing OS that happens to be the only one supported by online banking. It doesn’t even seem farfetched. I already am excluded from lots of software, games, and online services that require an OS I refuse to utilize, and the rate of that exclusion definitely seems to be accelerating quickly.
In my opinion some inline assembly would be a better spot fix than using volatile. You could create a few lines of assembly that express the exact intention here, i.e. round the value and store in RAM, versus using volatile which has a similar but essentially different intention. Using volatile may cause the bug to resurface in future compiler versions. This is how I would modify the function in question:
bool operator() (const double2& a, const double2& b) const
{
double da = deviation(a);
double db = deviation(b);
#if __i386__
// Make sure both doubles were rounded by forcing a load out of memory
// before comparing. This makes the comparison deterministic. If we
// don't do this, the compiler may optimize out storing the double into
// memory first causing some versions of the compiled code to compare
// 80-bits and some to compare 64-bits.
{
double dam, dbm;
__asm__("fld %1" : "=t" (dam) : "m" (da));
__asm__("fld %1" : "=t" (dbm) : "m" (db));
da = dam;
db = dbm;
}
#endif
if (da == db)
return a < b;
return da < db;
}
Not sure if there is a more efficient way to round an 80-bit register to 64-bit in x87.
What about using std::memcmp on the pointers of the two variables? Would that work? Unfortunately, I do not own a machine that reproduces the issue, so I can’t test it.
For sorting, yes memcmp() would work decently well. For implementing the equality primitive for a set type, it won’t give correct results when checking for -0 if +0 was stored in the set (and vice-versa).
For the record, before becoming a video game developer, I did a thesis in computational geometry, then worked for 6 years in a company that developed computational geometry software. In particular, I used libraries such as MPFR, which enable multi-precision computations to avoid double-related problems when computing on real numbers. Please believe me when I say that I know what I’m doing when I compare doubles.
With all due respect… The 80-bit intermediary representation and the problems it causes is quite elementary knowledge for those who want to rely on float determinism. That is exactly one of the reasons why you don’t do these equality comparisons on floating point numbers.
If you, too, saw the equality test on doubles and thought “lol, noob, of course equality tests on doubles don’t work”
Unfortunately, that initial thought turned out to be correct after all. There is a reason why alarm bells should go off in your head every single time you see a floating point comparison with ==.
The equality test on doubles is problematic only if you think you’re testing equality between real numbers (in the mathematical sense).
I have a hard time understanding this statement. The number 1 is a real number in the mathematical sense and 1.0 == 1.0 is true, so testing equality between real numbers in the mathematical sense is not necessarily problematic. Did he mean to say “irrational numbers”?
Anyway, problems appear whenever the number (be it rational or irrational) has enough significant digits to trigger rounding because there is not enough precision in the machine representation. There is a whole host of problems that can occur, for example floats also have denormals that can be turned on or off, rounding modes (and note that this stuff is thread-local state and can be modified by library calls), etc. Yes, IEEE-754 is deterministic in theory, but in practice, there are just so many optimizations and knobs to turn that it’s hard to get it to behave exactly this way.
A much more refined way (because you can integrate it only in potentially problematic places in your code, typically on double equality tests) is to use the “volatile” keyword.
Good god, please no. volatile is a keyword for things like device registers, not to work around floating point comparison issues. The correct solution is, of course, the recommended best practice: Using a tolerance, the very solution he initially rejected and that would have prevented this entire Odyssey. If the tolerance is chosen small enough, it will have pretty much zero impact on the rest of the algorithm and if it does have an impact, it’s almost always beneficial because it reduces noisy or undesirable effects caused by calculations at the precision limit.
With that being said, the game actually looks nice. I like adventure games, so I should try it out.
The 80-bit intermediary representation and the problems it causes is quite elementary knowledge for those who want to rely on float determinism.
I don’t have the same background as the author, but going into this article I also knew the FPU has an intermediate 80-bit representation. I remember learning about it and thinking huh, that probably improves rounding error in general, but it has some weird implications.
I was also surprised by the bug reveal.
Floats are complicated. It reminds me of cryptography: there’s lots and lots of details, and if you’re confident you got all of them right, it’s likely you’ve gotten one wrong.
The correct solution is, of course, the recommended best practice: Using a tolerance, the very solution he initially rejected and that would have prevented this entire Odyssey.
To be fair to the author, they took what I consider to be an alternate “correct” solution. Does this bug block shipping the game? No? Then ship the game! One approach plasters over floating-point rounding at the code level; the other, at the business level.
Though they probably would have saved themselves a bit of trouble if they had coded in a tolerance first thing. I’m (selfishly) glad for their odyssey, though, because that 80-bit representation is such a fun bit of trivia.
Unfortunately, comparing doubles with tolerance (i.e. abs(a - b) < eps) is also not an option here. If you do this, you may easily break the comparator properties for std::set (e.g. it turn out that a == b and b == c, but a != c), leading to another hard-to-debug UB.
Probably the only viable solution is to precompute the distances and store them in the set, rather than recomputing them each time the comparator is invoked.
Using a tolerance? Sorry, but you must not have read the code, or understood it at all. If, instead of comparing da and db, I do if(std::fabs(da-db)<epsilon), what happens? Well, if two vertices have a close deviation, instead of sorting them by deviation, we’ll sort them by their indices. GREAT. What’s the point, apart from making the algorithm less optimal?
The algorithm would simply be less optimal, apparently.
Anyway, you shouldn’t do sets over floating point values. There is a reason why this stuff isn’t even allowed in Rust unless you specifically use ordered_float. Rust floats don’t even have the equality trait Eq.
The algorithm would simply be less optimal, apparently.
I feel that it can become UB. Suppose you have eps = 1e-9 and added three points, a = 0, b = 5e-10 and c = 1.1e-9. Now, you have a = b, b = c, but a < c. Though, if you guarantee somehow that the similar cases cannot happen and you have total order, then it is OK.
There is a reason why this stuff isn’t even allowed in Rust unless you specifically use ordered_float. Rust floats don’t even have the equality trait Eq.
AFAIR the reason why Rust does this is the issue with comparing NaNs and is not related to precision or non-determinism issues.
Suppose you have eps = 1e-9 and added three points, a = 0, b = 5e-10 and c = 1.1e-9. Now, you have a = b, b = c, but a < c.
Good point! Damn, this stuff with floating point numbers in sets is even more dangerous than it seems at first glance.
AFAIR the reason why Rust does this is the issue with comparing NaNs and is not related to precision or non-determinism issues.
Hmm, I thought I saw a thread with an in-depth discussion about this and it was more than just the NaNs, Infinities and -0 vs +0. But I cannot find it anymore.
I really don’t understand what you think to accomplish by explaining the possible issues with comparing floats to someone who just explained they have very detailed knowledge of the possible issues with comparing floats and why in this case it was perfectly valid to do so, were it not for a compiler bug.
You really think you explained anything this person didn’t already know?
There is a reason why I have an iron rule for managing Ubuntu servers: Never make a dist-upgrade. It was a painful lesson to learn, it fails more often than it works. If you need to go to another release, throw away the old server and set up a new, clean one. Bonus points for validating that your procedure to set up the server from a clean Ubuntu image works.
I manage ~200 Ubuntu LTS servers since 2012 and have never had a dist-upgrade fail unless it exposed an underlying hardware failure that has been lying dormant, which appears to have been the problem here.
It really depends on what you have installed and how customized your system is. In essence, it’s a lottery and you seem to have been winning a lot, congratulations. However, there is a myriad of things that can go wrong:
APT index or packages are broken (especially if you use custom PPAs)
Old configuration files confuse new services or binaries
Dependencies become incompatible (e.g. too new libraries, output of binaries changes format or semantics, etc.)
Drivers become incompatible
Etc.
Life is better if everything is dockerized and your Ubuntu is just vanilla plus a docker installation.
It depends on how old your Debian was. I tried to upgrade an old Debian MIPS system that I inherited and discovered that the new packages required a newer version of apt. The newer version of apt needed a newer version of glibc. The machine became unbootable trying to get out of that mess. I believe newer versions of Debian ship a statically linked version of apt to avoid this problem.
Out of curiosity, did this happen during an N+1 upgrade, or was it a situation where the system went directly from several-versions-old to the most recent?
This whole thread feels like an advert for FreeBSD. When you run freebsd-update (to update the base system), it creates a new ZFS boot environment (snapshot of the root filesystem). No matter how badly it goes wrong, you can always revert to the old one (as long as you can connect to the console). After the update, you delete the old one. If it’s a major update, the new version will default to installing the the ABI compat components and so you can update packages later (and do the same kind of snapshot of /usr/local if you want to be able to roll those back). Doesn’t Ubuntu do something like this? I’d hate to do any kind of serious system maintenance without an undo button.
That’s a really clever strategy and no, I’m not aware of any Linux distro that does something like that. Guix and Nix get to an even broader reversibility with a very different strategy: the packages are managed like an immutable hash data structure in a functional language. The packages are the keys, and the files are left in-place on disk as the package is changed on-disk by creating a new set of symlinks (pointers) into them. There’s an eventual GC process for long-unreferenced data.
As an aside, this is so obviously the right way to do package and configuration management that I evaluate them ~yearly for use on Lobsters and personal projects, but the documentation/maturity aren’t there yet and I’ve never seen a working install of a Rails apps as they have significant impedance with language-specific package managers/apps distributed as source.
The root snapshot reminded me of transactional-update and ABRoot, which are (as best as I can tell, I haven’t used them) both snapshot-based strategies for applying updates.
I’m curious why this isn’t standard practice even for distributions that you do trust to upgrade versions correctly: spin up a new VPS with the new distribution, copy over (hopefully a small amount of) state, switch DNS to point to the new VPS. What are the arguments against that approach?
The argument in our case is that ansbile has been brittle. Some of that is investing more time in cold-start provisioning, but some of that has been ansible making backwards-incompatible changes more often than we make any changes to our playbook. I never wanted to spend the (it turns out) ~5 hours getting that working.
Also, “switch DNS to the new VPS” has bitten us, the new one was assigned an IP that’s on many spam blacklists. I didn’t think to check and reroll until getting a clean one (this is apparently the best practice for DigitalOcean).
Yeah keeping up with ansible changes is tiresome. I found that it’s helpful to use a scratch server to debug the playbooks without any time pressure.
When I was doing email, we ended up making some effort to keep the IP addresses of our mail servers as stable as possible. Rather than updating the DNS we would just move the service IP address(es) from the old server to the new one - these were virtual service addresses, in addition to the per-instance addresses.
Dunno how easy it is to fling IP addresses around like that in a VPS provider: probably hard or impossible! In which case I might treat the scratch server as a practice dry run for an in-place upgrade of the live server.
On Linode, you can swap IP addresses between machines. On DigitalOcean, it looks like Reserved IP addresses fill a similar niche. It looks like that would be for a new IP address, not the current ones, though.
Yeah, I brought this up last night. I was planning to file an issue or something like that when the current crisis had passed so we can do an orderly transition to a new IP and not have as many problems next time.
I’m chewing on this one. I’m a bit irked that the product is basically paying the hosting provider to avoid dealing with problems caused by inadequate enforcement against bad behavior by other customers. The most uncharitable version of this is that the company is extorting me to avoid the expenses of acting responsibly. So my hobbyist sensibilities about the way things Ought To Be are clashing with my professional sensibilities about how cheap an improvement to prod uptime it’d be. Probably I’ll get over things and set it up soonish.
Oh, I totally misunderstood the pricing and thought it was a flat amount per month. Thank you, I’ve added one to the web01 vps and I’ll transition DNS to it soon.
Disadvantage: You temporarily double the resource usage, as both the old and the new server exist at the same time for a short duration. Another issue is DNS propagation which is not instantaneous.
Overall, it is still my preferred method because if anything goes wrong, I notice it early and can keep things running on the old server while I troubleshoot the new, upgraded one. For example, the outage described above wouldn’t have happened if this procedure was followed.
Lobsters finances (my wallet) is fine with this kind of expense. Running Lobsters would be a cheap hobby at twice the price. Just wanted to post a reminder as people occasionally express concern about hosting costs, and I prefer not to take donations.
Considering the free nature of lobsters a $7/hr VM would quickly increase hosting costs even during cut overs if they things don’t go as planned.
The approach of running two VMs during cut-over doubles the cost only during the fraction of time that a whole-system upgrade is being made. In extreme cases that might be, what, 8 hours a year? That would add 0.1% to cost of the VM. I don’t see that it would “quickly increase VM costs”!
Consumer ISPs often have their own idea of TTLs independent what the authoritative server tells them.
Perhaps the correct approach is to use some sort of reverse proxy but if that requires a VM of its own it would definitely add a lot more cost.
What sort of costs do we expect to incur in consequence?
That depends very much on the server/plan. But yes, in the cloud, it usually costs peanuts. But if you have a beefy on-premise server, you can’t just double your hardware on a whim.
Would setting DNS TTL to 60 (seconds) sufficiently far in advance mitigate that disadvantage?
I don’t have enough experience with DNS management to know for sure. All I know is that this stuff keeps being cached at every step of the way and it makes life difficult sometimes. Unfortunately, it isn’t just a switch you can flip and it’s done.
There is a reason why I have an iron rule for managing Ubuntu servers: Never make a dist-upgrade.
That’s crazy. I’ve used “apt-get dist-upgrade” as the sole means of updating my Debian machines for 19 or 20 years now. Granted they’re desktops, and 90% of the time I’m going from testing to testing, but still, other than an Nvidia driver fubar once, it always works great for me.
If you need to go to another release, throw away the old server and set up a new, clean one.
The advantage of routine rebuild-from-scratch is that you get to practise parts of your disaster recovery process. When something surprising like a driver fubar happens, it does not happen on a production machine and it does not make a planned outage take much longer than expected.
I have also had no problems with Debian upgrades long-term. I’ve had two long-running Debian servers: one on physical hardware, installed in 2001 and dist-upgraded until I retired the machine in 2010, and one on a VPS, installed in 2013 and still running. Been very impressed on how it all just works. In the early years (2001-2006 especially) an upgrade would often break X11 and require me to futz with XF86Config, but that was eventually sorted out with better auto-detection. The underlying OS never had issues though.
I’m about 50/50 for do-release-upgrade working on my home servers. Failures are usually due to some weird configuration I did (hello, years of changing network configuration systems) or a third party package, I’m not mad about this, trying to support absolutely every possible Ubuntu system is impossible. But it is an issue for me.
Ah, the days when the entire campus shared a T-1 connection and it was just easier for UCB to mail you a 9-track of BSD 4.2 than to try and download it.
Apparently he wrote this in 1981 [1]. It’s amazing how over 40 years later, the premise still holds true. There seems to be a fundamental principle at play that could not be broken with technological innovations. Makes me wonder if there is a physical law that limits the speed of transfer of information depending on the mass and energy that is used to transmit it.
We’re moving a ton of data around these days, for $REASONS. I have repeatedly pointed to this example when people respond to the station wagon analogy with “that’s just something you oldsters say…that doesn’t matter now”. Suddenly, a dim light goes on (an LED, not a light bulb).
As that xkcd says, “all of that data is coming from somewhere, and going somewhere.”
To really hammer home how quickly the price of storage is still shrinking, the article suggests about 900-1000 USD for a 1TB SSD, but you can buy a 1TB Western Digital Black NVMe on Amazon.com right now for $50.
Perhaps? What’s the signal power of matter you take along? You just take e = mc²? I haven’t thought this through in detail, I was just wondering as I wrote the comment.
With digital data transfer you have to essentially take 1 electron at a time to the new destination.
With physical data transfer you can take trillions of electrons at a time to the new destination.
Your comment about e = mc² really does seem to touch this. We can only fling 1 electron near / at the speed of light at a time. So there’s a hard limit on how much data / second you can do. We can carry tons of electrons though at a time.
So we’re optimizing for “sending the least information that makes the most sense to us in the least time”, more or less. We don’t need to be sending each other 100GB videos, and we don’t. It’s much faster to send a 1GB video that is as recognizable as a 10GB video over the internet. It comes down to “we want to see something as fast as possible”. i.e. what most people say about this topic: optimizing for latency.
I think it’s better to see things as “digital objects”, like for example, if you wanted the Ethereum blockchain. It’d be faster to get it on a flash drive than downloading it for most people, but it can’t be broken down further than its “useful unit” i.e. itself.
100% awesome thought :)
Someone could probably graph when these thresholds cross? :o
With digital data transfer you have to essentially take 1 electron at a time to the new destination
Most bulk data transfer these days is over fibre. With modern fibre, you can send multiple wavelengths simultaneously. The upper limit on this is one symbol per photon, though I’m not sure that there is a theoretical limit on the number of distinct symbols. Practical limits depend on the discrimination of the receiver and the flexibility of the bends (different wavelengths travel at different speeds because they retract differently so unless you have a perfectly straight fibre and send to photons down perfectly straight, there are limits on how closely you can space them).
Similarly, flash doesn’t store one bit per electron. Each cell holds a charge and that encodes one symbol. With MLC, there are typically 4-8 possible values per cell. In theory, you could build storage where you use different energy levels in electrons on a single atom to represent this. Quantum mechanics tells you both how many possible symbol values you can store and why it’s really hard to build something useful like this.
For both forms of transfer, the practical limitations of the specific technology have much more impact than the theoretical limits.
It’s also worth noting that most of these comparisons ignore the bus speed on the removable storage. My home Internet can download (but not upload, because my ISP is stupid) data faster than I can read from cheap flash drives. Commercial fibre is faster that most local storage unless you are able to read and write a lot in parallel.
Yeah, the electron bit was just to simplify thinking around things, but you’re totally right. And good point about bus speed. I think the counter argument there is “you have the data at your disposal now”, so you don’t need to read it all off onto your primary storage device?
Last night before sleeping I basically concluded that if you can just add more wires (and I guess as you explained, wavelengths) then you can always beat the “practical carry limit” of physical data transfer… Except it seems you can always store more before you can transfer faster, at least that’s the trend right?
I’d argue the principle is Latency Bandwidth relation.
You can absolutely start pushing enough bandwidth to beat the pidgeon over most ordinary wire. But the protocol involved will mean your latency will suffer and normal usage will be near impossible.
For example, you could write a protocol that spends a minute probing out the exact physical properties of the transfer wires between source and destination so it can model the wire and figure out how to blast through a 100 gigabit packet of data once. The receiver will be spending that hour much the same trying to probe out the wire. Once the blast is complete you’d have to remeasure, as it’s now another minute and the wire might have warmed up, changing properties. Plus the receiver now needs to process this gigabit blast and ACK the reception (even just hashing it to make sure it got there will take a moment with 100 gigabits of data). Retransmissions will cost more time. But ultimately this incredibly horrific latency buys you a lot of bandwidth on average, not a lot in the median.
On the flipside, you can not do that and get much lower average bandwidth but much more usable latency guarantees, plus a better median bandwidth.
This is why it’s so important to word such things carefully. Going by the article’s title, you either don’t exist or are no one. In essence, the post discusses away the existence of certain people and mindsets. It is not only intellectually disingenuous and wrong, it’s also rude. It’s a complete non-starter for a fruitful discussion.
Eh. I think reading beyond the title is underrated. The question is “what are you willing to sacrifice to achieve simplicity?”, which requires some explanation but does lead to the conclusion in the title if the answer is, in practice, essentially nothing.
Why are you assuming that the only way to achieve simplicity is to sacrifice something? That is not true at all. A smaller simpler (but better) design can be more feature rush than a more complicated one.
Sadly, this type of wording in titles (also in videos) are the ones that get traction pretty much every where. It cannot allow for a middle ground in the point that (hopefully) the author is trying to move across.
There is no statement about everyone that can possibly be anything but hyperbole, other than the most basic statements of fact like “no one can survive without oxygen.”
I work on safety-critical real-time stuff. We can only use simple software otherwise it becomes impossible to guarantee safety and timing. For many use cases a Linux kernel is too complex. A microprocessor is too complex.
Author here. Obviously my post’s title was inflammatory hyperbole, but to try to defend my main point for a minute:
How many things in your web development stack can you name — things that you actually like and consider to be a Good Thing — that you would be willing to sacrifice to achieve simplicity?
Performance; as long as the code isn’t doing things in a boneheaded way, it’s almost always fast enough.
Visual effects; I’m cheating here – this isn’t something I actually like, so removing it is pure upside. But it’s complexity that many many applications insist on, and removing it would be welcome.
Abstraction; code is often shorter and easier to explain when written with fewer, smaller abstractions. The abstractions may be elegant, but they aren’t worth it.
Resilience; the ability to handle faults gracefully is often fragile, getting it right is time consuming. It typically doesn’t pay it’s way. Detect the error, crash loudly, and move on. Be resilient in a small number of carefully chosen chokepoints instead of spreading error handling everywhere. Aiming for five nines of reliability isn’t worth it.
Because we can’t be arsed to find another job, we let our manager convince us to add yet another baroque curlique to a product that should have been cast in concrete and declared “done” a decade ago….
…the only ethical thing to do actually is remove obsolete features and any remaining bugs.
But it’s still selling and unless we keep adding features we will be out competed by our competitors and creating a new product in a new category is too expensive and risky…
So day by day with a sick feeling in our stomachs we add yet another wart.
And our customers, being in a market for lemons, choose the glossiest and sparkliest lemon on the rack…..
The point, as I take it, is that “simplicity”, like “performance” and other good things people often claim to want more of, is a tradeoff: if you want more “simplicity” or more “performance”, you often will have to give up at least some quantity of something else you care about to get it. And reframing in this way is not just splitting hairs, because it leads to more productive discussions. For example, instead of “why don’t developers care about performance” (unproductive) we can reframe to asking about tradeoffs between performance, which developers do care about, and other things that they also care about, and how to affect the choices being made.
I do. And I installed it myself, which means I can claim with a straight face that dishwashers are pretty simple:
there’s a hookup for power. Electricity is used to heat water and run the controller which handles the user interface, sensors and timing commands. Finally, it runs a motor for a drain pump, possibly with a small grinder.
there’s a hookup for clean water input. Water is used to clean the dishes, and incidentally to spin the distributor arms.
there’s a hookup for grey water draining.
The primary differences between consumer dishwashers are the material of the central tub, the arrangement of the racks, and the arrangement, quantity and quality of the sound insulation. Most of the sound insulation is external and replaceable.
The primary differences between consumer dishwashers are the material of the central tub, the arrangement of the racks, and the arrangement, quantity and quality of the sound insulation
Apparently this is largely due unintended consequences to EU regulation on replacement parts for white goods. This meant that it’s not economically viable to produce dishwashers that are differentiated in any meaningful way unless you are certain to sell very large numbers. They all include almost all of the same components so that they can guarantee availability of replacements for the mandated period.
Once you understand it, it’s simple.
Having flooded the floor by not putting plumbers’ tape on when connecting up the water input, I can attest to the ‘once you understand it’ bit.
I don’t think reducing competition or innovation was an intended consequence. Policy makers being naïve and not considering second-order effects is fairly normal in my experience.
I of course don’t know what’s in anyone’s mind except my own, and you could well be right. I do think that regulatory capture is a transformation that most industries have gone through, and it would require a surprising level of naivete to not anticipate it. I am not an EU resident and I don’t really know any EU politicians, so I’m not really qualified to speculate I guess.
The API is simple, for you. The complexity is hidden behind the form of a commodity. Mining, manufacturing, transport, etc. It always seems like when programmers say they like when things are simple, they generally just mean “simple, for me”, and not simple, for thee. It seems like to me a proclamation for concerns to be handled by others.
All simplicity is subjective. if you disagree, I’m going to have to point out that you are made of billions of cells, each composed of millions of molecules, each built from up to a million atoms, each made of leptons and baryons and held together by forces that require conditions approximating the first minute of the universe even to observe reliably… so nothing is simple and the word has no objective meaning.
It’s important to note that PID controllers are a general concept and many embedded devices contain some implementations of PID controllers to control temperatures, timings, motor positions, etc. This is just one of many implementations of a PID controller. They make it sound like PID controllers specifically regulate temperature.
The intro of that article needs a link to Nicholas Minorsky who invented PID controllers for the automatic steering system of the warship the USS New Mexico. Rather a long way from a coffee machine!
When they installed new coffee machines at $WORK, I was astonished by what was exposed in the admin interface. The entire state machine for making different beverages is exposed, including temperature, flow rate, and (millisecond-precision) timing for each step. If you haven’t seen this, it’s well worth a look next time someone is servicing a machine. Apparently most of them have a sequence you can press that gives you a read-only view of the admin interface, so you can see exactly how your coffee is made, you just can’t change it.
I write software for commercial coffee machines (c++ and c) and us and most others have default codes for the service interface (try 1111, 2222, 12345, etc or lookup the service manual for your type online).
In my case the recipes are a sequence of steps, like, open this valve for x ms, run this grinder for y ms, move the brewer to position z, regulate flow).
I implemented PID code in the flow / pressure regulator for one of the brewer systems. With some special hardware it can auto correct to the requested pressure even if the coffee grind is too course or fine. It uses the flow data over time to give consistency, even if 6 months in your grinder has run in, theoretically giving a consistent beverage every time.
Go ahead and ask along if you want to know more.
(The main factory is in the Netherlands, but we also operate a Factory in Taylor, MI, for the American market, so you might even have seen a machine by us in the US)
Agreed. PID controllers are awesome. They’re an amazingly simple and general-purpose algorithm for any time you have a value and a target number, and you want to say “move the current value smoothly towards the target and keep it there even if the target moves around”.
move the current value smoothly towards the target
Perhaps even more interesting is you can purposefully tune it to not be smooth, if that’s to the benefit of your system. The smoother it is, the longer it takes to get to the desired value. It’s often advantageous to quickly overshoot and then rapidly back off. Basically oscillate towards the intended value while smoothing out over time.
Your statement is generally true, I just wanted to add this bit that I find interesting.
they are incredibly common in industrial applications. they can regulate flow, heat, cooling, anything really. they have cascading pid controllers and many other. PI or PD are also common subsets.
Apple is well-known for its walled gardens, security theater and hostility towards general purpose computing for decades already. If you buy these devices and give Apple your money, you deserve what you get. You’re not in control when operating these devices. If that’s what you want, fine. If that’s not what you want, don’t buy Apple.
Walled garden, sure. But Apple has top-notch security and it is absolutely unfair to claim otherwise. Out of any platform available for the general populace, it is by far the safest device to own (in part due to its walled garden, sure) — from the hardware all the way to software. The only other device that is similarly well-protected is a Pixel with GrapheneOS.
I do like and use desktop linux as much as the next person, but we should all know that an npm install could literally do anything on your computer from encrypting your hard drive to installing a permanent keylogger, the only thing protected is installing a video driver, as per the old xkcd, and unfortunately there is still a certain mindset that everything is security theater (though it has improved over time, back in the days even Linus and the kernel were prone to this, until proven otherwise. Nowadays the problem is mostly userspace).
But Apple has top-notch security and it is absolutely unfair to claim otherwise. Out of any platform available for the general populace, it is by far the safest device to own
Citation needed. This is an often repeated claim for which there is zero evidence. When’s the last time a vanilla Debian installation was pwned by zero click exploits, as it happened countless times on iOS over the past couple of years? My falsifiable claim is that an ordinary machine with latest Debian stable is way more secure against digital attacks than any Apple deployment.
As for npm, makefiles, and so on: You are a developer, you know the risks: Vet the code or run it in a VM. It’s not rocket science (and yes, I know, VMs are not impervious, but they exponentially raise the difficulty of exploiting the device).
Don’t get fooled: Great security is compatible with user freedoms. The big corporations make people believe that these two things are mutually exclusive, but the opposite is true: A lack of user freedoms is a severe security risk by itself. A prime example are locked down Android devices bundled with uninstallable crapware.
Apple systems are safe when you define safety as the user not being in control of the device. To me, that is the opposite of safety - a device is safe when I can install and remove whatever I want. It is high time we stop allowing monopolists to define the term safety as they please.
Citation needed. This is an often repeated claim for which there is zero evidence. When’s the last time a vanilla Debian installation was pwned by zero click exploits, as it happened countless times on iOS over the past couple of years? My falsifiable claim is that an ordinary machine with latest Debian stable is way more secure against digital attacks than any Apple deployment.
That’s a fairly apples to Debian’s comparison. Last time I installed Debian, it didn’t include a GUI or a web browser. It did include the Linux kernel, which has had quite a lot of vulnerabilities. Just looking at the ones in the last month, there are a lot of use-after-free bugs in the USB stack. If the same bugs existed in XNU, they would be less easy to exploit because XNU uses kmalloc_type, which does type-pooled allocation, so none of these bugs would allow an attacker to get a type confusion bug and trigger exploits.
If you do get a root compromise on the Debian system, out of the box it doesn’t provide any secure boot, so it’s trivial for an attacker to modify the kernel binary image on disk and have persistent root access. In contrast, iOS and macOS have a full secure boot chain. iBoot uses a special compiler mode that provides memory safety (and a reasonable degree of type safety) for the code, whereas GRUB (the equivalent on a Debian install) had a memory safety bug last year that let you bypass secure boot and required all cloud providers to roll out updates that revoked the signing keys. Once you boot, the filesystem layout that Apple inherited from NeXT has a clear separation of system files and modifiable files, which makes it easy for Darwin-based systems to have an immutable base image, whose hash is part of the secure boot chain. This means that a compromise can’t modify any system files (though it can put alternative versions in your path). This makes cleaning up from a compromise much easier, because you have a large set of known-good things to use.
Beyond that, if you install a random app via apt-get, it will (with the exception of a very small number of things that come with SELinux policies) run with the full privileges of the user that runs it. If it has a vulnerability that an attacker can exploit, it can compromise all data owned by the user. In contrast, anything installed from the app store is sandboxed by default (with Apple’s version of the TrustedBSD MAC framework) and has no access to the filesystem except for its own private region and files that the user explicitly grants access to. Even apps installed from random sources on macOS often now come with similar policies. For example, UTM on Mac runs sandboxed and so a VM escape is limited to the files that the app has access to unless it also has a sandbox escape.
When’s the last time a vanilla Debian installation was pwned by zero click exploits, as it happened countless times on iOS over the past couple of years?
When was the last time a nation-state level threat actor wanted to attack vanilla Debian installations? For Android/iOS, there are companies which exist just to find these vulnerabilities to make exploits that are sold to nation-states, since cell phones are big targets for repressive regimes. Some examples are Hacking Team and NSO Group, Are they attacking Linux installs?
Most servers on this planet run some Linux flavor. Debian and Ubuntu (which is Debian based) are extremely widespread. They are certainly attacking Linux installs beyond Android, yes. Anyone who maintains servers with exposed ports on the internet knows that there’s constant probing and attacking going on.
You are comparing a server maintained by someone at least a bit knowledgeable about computers that usually run self-made, trusted programs and are otherwise completely sealed off from any kind of external environment, to a device that can be used by my grandma to install any kind of untrusted app, right next to all the family photos, credit card, etc.
It’s not hard to protect a single port of access, as opposed to a mobile’s huge surface area.
It’s not hard to protect a single port of access, as opposed to a mobile’s huge surface area.
Server security can actually be incredibly difficult and the surface area is often larger than what is exposed on a personal machine. Also, servers can be and often are very high value targets.
Either way, the discussion has gone full circle. If we as a society let large corporations decide what code can and cannot be executed, we the people will experience a loss of rights and powers the likes of which has never been seen before in history. And precisely this is happening at an accelerating pace. As an example, it has now been proven beyond doubt that the US government pressures social media companies into censoring uncomfortable speech and political opposition (prime example: Hunter Biden laptop). These walled gardens are the perfect instrument to impose a wide variety of restrictions quickly and easily. By the time people realize that the security argument was a Trojan Horse, it’s too late.
There is no security reason why Apple’s or Google’s keys must be in the TPM and you cannot put yours in there to replace it. Is granny going to generate an asymmetric key pair to customize the system? No, she isn’t. Again: Freedom and security are not incompatible, they belong together.
It is remarkable how many people with technological expertise fall for this cheap PR. Pay 99$ to protect me… From myself? Preposterous. It should be called out for what it is: A grift, rent seeking, abuse, exploitation. And it should be boycotted.
Same thing with Web Environment Integrity. It has nothing to do with security and everything to do with control and power. No profit driven company cares about you being secure beyond the letter of the law, don’t make me laugh. One thing is certain: We will hear a lot more about remote attestation in the near future and it’s not going to be pretty.
because less people use gnu/linux on personal computers? but (this has happened)[https://www.vice.com/en/article/v7gd9b/facebook-helped-fbi-hack-child-predator-buster-hernandez] on debian-based systems as well
I don’t understand your comment - my point is that Linux desktop has practically no security whatsoever: any program (e.g. npm) runs with the same privileges as your file browser, so the UNIX model is basically useless at that granularity. There are solutions, e.g. Android makes the whole thing secure built on the same base, but mainstream linux userspace barely cares about it.
OSX is a lot better, probably the best out of the 3, but ios is better.
I think this is a joke but I use bing to generate dall-e 2 prompts - and claude for midjourney, I get better results than manually writing image prompts.
I don’t know any off the top of my head for PlantUML specifically, but there are definitely tools that will generate UML diagrams from existing codebases (though I imagine you’re more interested in sequence diagrams and such)
I have no clue what the author is looking for. Querying fonts with GTK (and all other GUI toolkits that use Pango, which is extremely widespread) is simple: https://docs.gtk.org/Pango/class.FontMap.html
One thing you can theoretically sensibly monitor your logs for is specific, narrow signals of things of interest; for example, you might monitor Linux kernel logs for machine check error messages. The first problem with monitoring this way is that there’s no guarantee that the message you’re monitoring for won’t change. Maybe someday the Linux kernel developers will decide to put more information in their MCE messages and change the format.
Note that this is something that has already been addressed with the Printk Index. A good article can be found on LWN.net, of course.
Yes, in Open Source, there’s no WONTFIX. There’s just “you have to do it yourself” ;-)
When I started doing Open Source contributions, I was surprised to see that pretty much all my PRs were accepted into various projects for which I wrote fixes and improvements. That was really motivational - I expected the reactions to be a lot more apathetic or even hostile. I’m sure such a patch would be welcome and eventually find its way into the upstream repository.
I miss the time when all the fonts were nice and crisp. It’s so hard to get modern systems to respect that without the fonts looking utterly terrible. These little pixel fonts have more artistry and craftsmanship in them than all those fancy vector graphics fonts we blur and anti-alias so carelessly.
Ironically, the practice of disabling anti-aliasing for CJK fonts / systems by default is very widespread due to the fact that these fonts are complex enough that you must make every pixel count and you can’t just vomit vector graphics all over the pixel grid. This is why you see Asian screenshots with crisp text even though ours is blurry.
For those of us who care for monospace uses Terminus is available at sizes up to 32 pixels so it works well at 4K. Lucida Typewriter is another choice (up to 34px) but the commonly available version doesn’t have the zeroes dotted. Any others?
Yeah, it’s only up to 12px. Well, I guess 24px is just 12px upscaled x2, but you miss out on the opportunity to use those additional pixels for more well-rounded characters.
It’s not the blurriness that’s the problem for me; it’s the consistency. On a 4K screen I don’t notice, but anything less it bothers me if e.g. m and n have different weight due to different positioning relative to the pixel grid.
We need to tell people that literally anything they read or hear can be wrong. It’s high time that the majority of people gets this memo - in fact, it’s of civilizational importance. The “confident bullshitting” that ChatGPT exhibits is nothing compared to what reaches our ears all day every day from news and politicians. The damage caused by unfounded belief in what conmen say is untold - ChatGPT is a minor offender in the grand scheme of things. With a bit of luck, hallucinating AI will usher in a second age of enlightenment by forcing people to practice basic epistemological hygiene. Such a cultural development might be as impactful on human wellbeing as the practice of literal hygiene by washing hands.
I have spent a lot of time thinking about the nature of science and epistemological hygiene and wrote some posts about it over the years:
I believe that the ChatGPT issue is downstream from this. You can teach a computer to repeat what is commonly understood to be true, but you cannot teach it to tell the truth - and those two things are further apart than you might think. ChatGPT makes obvious an epistemological horror that has always been present - someone confidently telling you nonsense and you believing it. This problem is as old as language and the remedy is a set of intellectual tools to sort out incorrect information, taught to everyone. You can make ChatGPT a bit better, but that won’t solve the root problem. ChatGPT two millenia ago would have taught you the geocentric model.
This seems a tu-quoque flavor of moral relativism. All peer reviewed studies are useless to me unless I have personally reviewed them myself? Some scientists have written some wrong things some of the time, therefore, all scientists and ChatGPT are equally unbelievable?
This seems a tu-quoque flavor of moral relativism.
To the contrary: It would be epistemological relativism to blindly trust certain individuals or publications. Everything has to be read with doubt in mind. And every piece of information has to be investigated for internal contradictions or contradictions with known facts, that is the bare minimum to steel ourselves against incoherence and confusion, which are ready to enter our lives at every turn. If people learned and cultivated this skill, they would immediately see through ChatGPT’s fabrications, among other things.
All peer reviewed studies are useless to me unless I have personally reviewed them myself?
I never claimed that. Peer reviewed studies are an important source of information. But just like every other source of information, they are by no means fully reliable. Our scientific institutions are plagued by issues like the replication crisis. Just recently, the Stanford president had to announce his resignation because it was uncovered that he was the principal author of papers with manipulated data. That’s just one of many examples of fraud in science. Just because they perform their work in historic buildings, add titles to their names, call their essays “papers” and publish in esteemed journals doesn’t necessarily mean that there is any truth in their work.
Some scientists have written some wrong things some of the time, therefore, all scientists and ChatGPT are equally unbelievable?
What an awfully uncharitable way to twist my words. I get the feeling that you were somehow emotionally put off by what I wrote, which lead you to construct straw-man arguments. My point is that people should vet and filter incoming information, no matter the source. Everyone who unquestioningly believes what ChatGPT writes was already intellectually deeply troubled before ChatGPT existed, for they let the garbage flow unfiltered into their mind and worldview. The problem is that we live in a society where people need to be repeatedly warned that not everything they read on the internet is true, a fact that should be painfully obvious even to the feeble minded.
If anyone else is planning to create new software that relies on NFSv3: please don’t. 32-bit timestamps are part of the protocol which will run out in the year 2106. There are people alive today who will have to clean up that mess.
You seem to be rather pessimistic about the development of retirement age.
Or, alternatively, rather optimistic about the likelihood of our surviving until then.
Or also optimistic about how soon people will start working on fixing the issue, instead of waiting to the last decade :)
This reminds me of when I was a child learning BASIC. I drew a circle to the screen and thought “so this is it. A perfect circle. Wow. It’s a little blockier than I thought, but it’s on a computer, so it must be perfect, right?”
I distinctly remember the moment, when I was 12, that I realised that the opposite of such things was true.
I wrote a BASIC program to repeatedly take the square root of a number, and graph it. Eventually, that result became zero, as opposed to doing what I expected it to do - approach zero. I was deeply confused at the time.
Edited: I was surprised to find as I aged that some of my most vivid memories of childhood are confusion and discovery. And how many of them relate to computers :)
For me, it was logo. We were taught to write programs to draw equilateral triangles and squares and told to draw regular polygons with 5, 6, and 7 sides. After a few tries (I was 7 at the time and had only been programming for a few hours), I wrote a program that generated an n-sides regular polygon. I tried it with 500 and it (very slowly!) drew a circle. I knew about pixels, so I knew it wasn’t really a circle, but realised that a circle was an infinity-sided regular polygon, which felt like a major revelation. It wasn’t until I was about double that age that I learned the maths that explains why that’s an interesting conclusion.
A computer can certainly represent a perfect circle (though it cannot display it). It can also represent the square root of a number, taken as many times as you would like.
On the flip side of the coin, we can formalise what computers do when they do approximate; the computer is then a perfect or near-perfect rendition of these formalisations.
Ah, yes. That was almost my first experience with Common Lisp, and it convinced me to persist with learning the system.
That right there is a language doing the right thing ;)
I was upset when I took a summer class that taught us C and the instructor told us about bit precision. It’s a computer, what do you mean it can’t just store all the bits in 0.1!
Yeah. As a kid, it took me a while to understand the difference between the discrete world of digital machines and the continuous world of some mathematical concepts. I heard the term “discretization” a lot but I only truly grasped it after I had more programming experience. The instant feedback of modern programming really accelerates learning.
The slowdown has nothing to do with iterators. And it has nothing to do with Rust. At the core, it’s a very basic and fundamental concurrency bug. As he mentioned in the post, if the programmer refactored the two
forloops into a single one instead of an iterator, the exact same thing would have happened. Conversely, if he used two separate iterations, the bug wouldn’t have happened. And of course the situation would be exactly the same in C++ or any other language like that. This is bordering on clickbait, really nothing to see here. I expected a post addressing real performance costs of iterators.The author goes by she/her pronouns btw
You’re assuming bad faith for what I read as an example how someone would naively (intuitively?) convert a to b, and it’s in Rust and it uses iterators - that’s what happened. Of course this can happen elsewhere, still nothing in this post is wrong.
But yeah, one could probably frame it a little bit different,
Yes, it does. The post clearly states that this is a problem for beginners, and I think you’re just to knowledgeable to even understand the issue. Of course this needs two loops, but the problem is that a beginner might not understand that .map and .for_each get turned into a single loop. In a language like js or ruby, stuff like this would run as two loops. That’s the entire misunderstanding.
This isn’t clickbait, but you aren’t the target audience.
Actually, I hear that iterators actually tend to be slightly faster than
forloops in Rust.Many years ago, I was at a talk by Robin Milner where he argued that artificial intelligence is the wrong goal and that we should focus on augmented intelligence. Making computers do things that humans can do is not especially useful, making computers and humans able to do things that neither can do in isolation is hugely beneficial. Things like Copilot fit well into that lens, they don’t replace the human they let the human accomplish more.
To the notes at the end, I don’t find it worrying that an LLM can reproduce college essays, at least in the sense that’s implied. I see this as evidence that these forms of assessment are not measuring the right thing. I hope LLMs will be a disaster for the kind of assessment that doesn’t require original thinking yet purports to measure that faculty.
I agree. When these LLMs reach such high scores on standardized school/university tests, the shocking thing is not how far computers advanced, but how primitive our assessment of skill truly is. That it can be fooled by what amounts to a probabilistic mechanism that generates likely chains of tokens. That is not the kind of skill and resourcefulness you look for when you’re hiring people. True resourcefulness is navigating uncharted waters, not doing what is familiar but doing what is logical, given the circumstances.
It’s tricky. When you design an assessment there are four key things you need to think about:
A lot of university exams don’t even start with a construct against which they can measure validity. They have some aggregate hand waving towards impact (do people who graduate our course get good jobs and do well in them?) but often the sample sizes are too low. They do care about reliability because that’s the kind of thing that they need to demonstrate for accreditation, but only very loosely. They care a lot about practicality because the people marking assessments are not doing that full time, they’re also doing research.
Even with those caveats, a number of the things where LLMs do well are the kind of assessments that are intended to drive learning rather than give a final score. They are intended to tell the student what they need to study more. Cheating at these is like cheating at exercise.
I wasn’t at the talk but I think I saw either a recording of it, or he made that point in a later talk that I saw, and it was a pretty defining moment for my outlook on programming, and on technology in general. Based on that, and on my personal observations, I also think rumours of programming being on its way to the urn because of “AI” (by which commenters usually mean LLM) are greatly exaggerated. This article is probably a good example: it has a few good ideas, but the examples are tone-deaf.
I don’t remember where I read it first (I think Norvig quotes it in his big book on the topic, but I mostly skimmed it – my interest in AI is mostly confined to writing things that allow people who are better at math than I am write AI tools) but at some point, someone noted that mankind finally achieved its ancient dream of flying when it stopped trying to emulate the flight of birds, and tried to make things that fly starting from the principles of flight but without attempting to emulate the form of flight.
The examples in this article miss that: they get hung up on code monkey “tasks” (which may ultimately be fixed by incremental progresses in LLMs; ChatGPT 4 fares better on a lot of things, who knows what ChatGPT 6 will be able to do) or pointless programming puzzles. With any luck, something good may come out of that, like the death of interview programming puzzles, but that’s IMHO that’s not where the potential of LLMs lays.
I’m (slowly – the web is really not my platform) writing a tool that uses LLMs for very practical problems. tl;dr wife runs a marketing agency, I’m writing a tool that assists with the analysis of various types of promotional text. It’s very much an analysis tool; it has an interface that vaguely alludes to virtual assistants but the interaction is not chat-driven at all.
After hacking at it for a couple of evenings I’m fully convinced that LLMs will not spell the end of programming, and I’m really enthusiastic about what they can do:
The examples in the article are relevant in one way – I’ve found LLMs to be remarkably good at doing things that otherwise require tedious and somewhat “machine-like” human effort. For example, despite extracting and correlating a bunch of data from freeform text, and then showing it in standard format, my code has basically no serialization/deserialization logic. If I need ideas extracted from a text, and I need to hand them off in JSON format to the UI layer, I just request them in JSON format. If I need to show formatted text, I just request it in Markdown format. It’s not foolproof: chatgpt-3.5 occasionally hands me invalid JSON or tries to be nice and say “Here’s your JSON first”, so there’s some basic sanitizing logic in there; and there are security implications to it that I haven’t fully come to terms with. However, these look like problems that are perfectly tractable between incremental improvements in LLM (or, for that matter, some format awareness) and some real-life experience.
This part of writing an application may go the way of the Dodo – all that code that takes these two JSON objects and produces this other JSON object in a slightly different format, or takes this CSV document uploaded by the customer and produces this standard BOM table. But bloody hell, good riddance, if anyone dreads a future in which we’ll have to do less CRUD and object mapping, they’re suffering from Stockholm syndrome.
Seeing how prescient Milner was in general, I’ll go ahead and agree with him.
I think things could head in this direction? LLM-generated code is probably difficult to trust. There is some interesting work on synthesis based on contracts / advanced type systems using SAT/SMT. I expect both approaches might converge. We may end up designing programs, with AI helping to fill in the gaps.
Some things are. This is explicitly the goal of copilot. I am not convinced that it’s going in the right direction though because it’s acting as a glorified autocomplete and if typing is the bottleneck when programming then I’m doing something wrong. The kinds of things that I would love to see from LLMs are:
The main thing here is that they’re all low stakes: it won’t break the code if the LLM is wrong (except the refactorings, but that can be coupled with testing) but it will make the output much nicer for the same human investment.
assuming that the human brain is not the maximum possible intelligence a mind can have, and that artificial intelligences can surpass our intelligence by orders of magnitude, and that they can be run on faster processors with larger data stores, that AIs could be reproduced with a simple ctrl-v ctrl-p– the question should not be “what is AI good for?”, but “extrapolating current trends, how much longer will humans be useful for anything at all?”
That is probably true, though it may be that it devolves into discussions about forms of intelligence. The human brain is an interesting balance of compute vs I/O and it may be that increasing compute power alone is not useful, increasing compute and I/O gives something that is unrecognisable. Given that we currently have a sample size of one for building models of kinds of intelligence, there’s a lot of margin for error in anything that we build.
That’s a nice hypothetical but no one knows how to build one that even reaches the level of a human yet. The article shows problems that young children can solve but that state of the art LLMs can’t. To exceed by orders of magnitude, they’d first have to reach parity.
That said, there are some things that computers can already do orders of magnitude faster than humans. They can compute the sum of a list of integers many orders of magnitude faster than humans, for example. A human with a spreadsheet can achieve far more than a human without a computer, but the spreadsheet by itself can achieve nothing without human agency framing the problems. There are a lot of places where human+computer could be vastly more useful now than they are, but are being overlooked in the rush to make computer-without-human systems.
Again, that’s also not clear. Human brains are probably not quantum computers, but they are highly connected neural networks. Neurons in the brain can connect to several thousand other neurons. Transistors on silicon chips can connect to single digits of other transistors. Simulating this with matrix multiplication is not space efficient and slows down as the connectivity grows (especially when you use back propagation to represent feed-backwards neural networks). Adding larger data stores increases latency.
So, your argument starts with three axioms, none of which is proven and at least one of which may not be true in the sort to medium term. Logical arguments that start from axioms that are not generally helpful.
Extrapolating from current trends, forever. That may not be good extrapolation because this kind of development often happens in bursts. Extrapolating from the difference engine, humans would be useful for menial calculation for a long time, but extrapolating from the first electronic computers gave a clear end point for that utility. AI systems running on quantum computers may be able to express forms of conscious intelligence that are so far beyond humans that they’re unrecognisable, given some clever bit of mathematics that no one has thought of yet.
I think that you’ve begged the setup. What is intelligence? Or, at least, what does it mean to compare intelligences? Is there a lattice of intelligence?
Does this still use their own analyzer or did they switch to
rust-analyzer?It uses JetBrain’s own analyzer implemented in Kotlin. I have been thinking whether its best for JetBrains to do their own thing, or to use an existing project (racer -> rls -> rust-analyzer) since 2015 and still don’t know the answer! For ecosystem as a whole, diversity is clearly good.
The great thing about Rust’s license is that they can fork and add proprietary enhancements to rust-analyzer if they want.
Great or terrible, depending on your viewpoint.
The rust community seems to prefer licenses that encourage proprietary forks. I imagine they’d love this.
Why would a community founded on open source “love” closed source software to the detriment/obsolescence of that open source software JB provided previously?
They chose licenses that were designed to allow proprietary forks. The MIT license was written for the X Windows system that was intended to be (and was) forked by proprietary UNIX vendors to build a compatible ecosystem of proprietary tools. I presume the Rust folks are excited to see this happening in their ecosystem. If they’d wanted a different kind of ecosystem they’d have chosen a different license.
The software in question is the official system used by Dutch municipalities for tallying the total number of votes after collecting them from individual voting stations. This used to be done with some sort of Excel template, and this has been in the news before for being a pretty iffy way of going about things, after which this particular software was created. So you could say this was quite a serious leak. Elections are coming up in November, so the timing of this is pretty good too.
According to the NOS (a Dutch news organisation), security audits had been done, but the audits did not include the installer. Future audits will include it. This points out the flaw of trusting too much on audits, and not taking a wider picture approach where the entire software development process is audited.
I’m quite surprised that there is an official system for doing this. In a paper election, each candidate (or their party) can provide monitors who watch the count. I’d expect an open electronic system to use multiple, independently developed, systems for this and require that they agree within the margin of error for an election to be certified.
The counting of votes is done at the individual voting stations, and the general public has the right to witness this counting. This system is used for adding up those totals from the stations so that they can be reported “up the chain”, which AFAIK doesn’t happen in public view.
So if I want to attack a Dutch election, this looks like the best place to do it. Having two independent implementations of the aggregation would help a lot there because I’d have to compromise both. Allowing parties to provide their own aggregation software would also help, because then if they disagree with the official tallies you can introduce an audit.
Electronic vote counting seems like such a dangerous and irresponsible thing to do, especially for countries that do not produce their own silicon. I can’t believe that it’s such a widespread practice, given the obvious dangers that are well-known for decades already. It’s so much harder to fake entire paper trails at scale.
Sounds really bad. This will make it hard to run non-mainstream platforms.
I guess regulators might step in and demand alternative modes of attestation, like they have done in case of iOS and the single-store monopoly. However, they are exceedingly slow at doing so. For instance, now it is impossible to do online banking in some places if you don’t run iOS or Android. Something as innocent as AOSP is not supported, even if you run the same banking app, due to the lack of SafetyNet attestation.
It’s also not clear to me that policymakers have any particular reason to care about non-commercial platforms or software modification as civil liberties issues. So we might not get the regulation we want…
Honestly not a bad time to start thinking about what the rules OUGHT to be in an ideal world. Even if we can’t get there anytime soon, knowing where we want to go has to help somehow.
The EU has a good reason to care. All of the big OS vendors are based in the US. If things require an attestation that only a US company can provide then this makes it impossible for any EU startup to ever grow to being a big company in this space.
This is especially true when you consider that things like the YouTube and Android are owned by the same US entity. If Google decides to require attestation to watch YouTube videos (so that they know that you aren’t using a client that skips ads, for example) then there is a fairly clear argument under existing monopoly law that they’re using their effective monopoly in one market to ensure that there can be no competition entering another market where they’re a significant player.
I think we need to learn a lot from the problems with DRM here. We pushed back on the technologies, rather than the uses. We didn’t block the technology in standards bodies (in part because there are some non-evil uses, such as access control within an organisation) and we didn’t get regulations on acceptable use. I’d love to see using DRM that prevents users from exercising fair use rights treated as vigilanteism and grounds for having copyright on the work in question revoked. Similarly, anyone using this kind of attestation to lock in clients to a particular platform should be subject to serious antitrust investigation.
The political leaders of the EU have a good reason to care that European keys find their ways into the TPMs of digital devices. They have no reason whatsoever to care about civil liberties of Europeans, or that people can run their own libre software on their devices. Quite the contrary - free computing is a threat to governing bodies, especially those that like to grow in size and power. The TPM menace will only grow worse as time goes on.
Yes, absolutely. Good thoughts.
My immediate vision is of a future where I end up having to literally drive to the bank to pay my bills because, eventually, I will no longer run a rights-infringing OS that happens to be the only one supported by online banking. It doesn’t even seem farfetched. I already am excluded from lots of software, games, and online services that require an OS I refuse to utilize, and the rate of that exclusion definitely seems to be accelerating quickly.
Yes, agreed. Quite distressing, but that’s the path we’re on and the route I’ll take as well.
In my opinion some inline assembly would be a better spot fix than using
volatile. You could create a few lines of assembly that express the exact intention here, i.e. round the value and store in RAM, versus usingvolatilewhich has a similar but essentially different intention. Usingvolatilemay cause the bug to resurface in future compiler versions. This is how I would modify the function in question:Not sure if there is a more efficient way to round an 80-bit register to 64-bit in x87.
What about using
std::memcmpon the pointers of the two variables? Would that work? Unfortunately, I do not own a machine that reproduces the issue, so I can’t test it.I believe that works for all pairs of non-NaN values except (+0, -0).
memcmp()on floating point numbers will give you reasonably sensible results ifFor sorting, yes
memcmp()would work decently well. For implementing the equality primitive for a set type, it won’t give correct results when checking for -0 if +0 was stored in the set (and vice-versa).With all due respect… The 80-bit intermediary representation and the problems it causes is quite elementary knowledge for those who want to rely on float determinism. That is exactly one of the reasons why you don’t do these equality comparisons on floating point numbers.
Unfortunately, that initial thought turned out to be correct after all. There is a reason why alarm bells should go off in your head every single time you see a floating point comparison with
==.I have a hard time understanding this statement. The number 1 is a real number in the mathematical sense and
1.0 == 1.0is true, so testing equality between real numbers in the mathematical sense is not necessarily problematic. Did he mean to say “irrational numbers”?Anyway, problems appear whenever the number (be it rational or irrational) has enough significant digits to trigger rounding because there is not enough precision in the machine representation. There is a whole host of problems that can occur, for example floats also have denormals that can be turned on or off, rounding modes (and note that this stuff is thread-local state and can be modified by library calls), etc. Yes, IEEE-754 is deterministic in theory, but in practice, there are just so many optimizations and knobs to turn that it’s hard to get it to behave exactly this way.
Good god, please no.
volatileis a keyword for things like device registers, not to work around floating point comparison issues. The correct solution is, of course, the recommended best practice: Using a tolerance, the very solution he initially rejected and that would have prevented this entire Odyssey. If the tolerance is chosen small enough, it will have pretty much zero impact on the rest of the algorithm and if it does have an impact, it’s almost always beneficial because it reduces noisy or undesirable effects caused by calculations at the precision limit.With that being said, the game actually looks nice. I like adventure games, so I should try it out.
I don’t have the same background as the author, but going into this article I also knew the FPU has an intermediate 80-bit representation. I remember learning about it and thinking huh, that probably improves rounding error in general, but it has some weird implications.
I was also surprised by the bug reveal.
Floats are complicated. It reminds me of cryptography: there’s lots and lots of details, and if you’re confident you got all of them right, it’s likely you’ve gotten one wrong.
To be fair to the author, they took what I consider to be an alternate “correct” solution. Does this bug block shipping the game? No? Then ship the game! One approach plasters over floating-point rounding at the code level; the other, at the business level.
Though they probably would have saved themselves a bit of trouble if they had coded in a tolerance first thing. I’m (selfishly) glad for their odyssey, though, because that 80-bit representation is such a fun bit of trivia.
Unfortunately, comparing doubles with tolerance (i.e.
abs(a - b) < eps) is also not an option here. If you do this, you may easily break the comparator properties forstd::set(e.g. it turn out thata == bandb == c, buta != c), leading to another hard-to-debug UB.Probably the only viable solution is to precompute the distances and store them in the set, rather than recomputing them each time the comparator is invoked.
In the case here, I believe it would not be UB:
The algorithm would simply be less optimal, apparently.
Anyway, you shouldn’t do sets over floating point values. There is a reason why this stuff isn’t even allowed in Rust unless you specifically use ordered_float. Rust floats don’t even have the equality trait Eq.
I feel that it can become UB. Suppose you have
eps = 1e-9and added three points,a = 0,b = 5e-10andc = 1.1e-9. Now, you havea = b,b = c, buta < c. Though, if you guarantee somehow that the similar cases cannot happen and you have total order, then it is OK.AFAIR the reason why Rust does this is the issue with comparing NaNs and is not related to precision or non-determinism issues.
Good point! Damn, this stuff with floating point numbers in sets is even more dangerous than it seems at first glance.
Hmm, I thought I saw a thread with an in-depth discussion about this and it was more than just the NaNs, Infinities and -0 vs +0. But I cannot find it anymore.
I really don’t understand what you think to accomplish by explaining the possible issues with comparing floats to someone who just explained they have very detailed knowledge of the possible issues with comparing floats and why in this case it was perfectly valid to do so, were it not for a compiler bug.
You really think you explained anything this person didn’t already know?
There is a reason why I have an iron rule for managing Ubuntu servers: Never make a
dist-upgrade. It was a painful lesson to learn, it fails more often than it works. If you need to go to another release, throw away the old server and set up a new, clean one. Bonus points for validating that your procedure to set up the server from a clean Ubuntu image works.I manage ~200 Ubuntu LTS servers since 2012 and have never had a
dist-upgradefail unless it exposed an underlying hardware failure that has been lying dormant, which appears to have been the problem here.It really depends on what you have installed and how customized your system is. In essence, it’s a lottery and you seem to have been winning a lot, congratulations. However, there is a myriad of things that can go wrong:
Life is better if everything is dockerized and your Ubuntu is just vanilla plus a docker installation.
I’ve never had a Debian version upgrade fail on me yet. I guess Ruby version upgrades are always going to be painful though?
It depends on how old your Debian was. I tried to upgrade an old Debian MIPS system that I inherited and discovered that the new packages required a newer version of apt. The newer version of apt needed a newer version of glibc. The machine became unbootable trying to get out of that mess. I believe newer versions of Debian ship a statically linked version of apt to avoid this problem.
Out of curiosity, did this happen during an N+1 upgrade, or was it a situation where the system went directly from several-versions-old to the most recent?
It might have been multiple versions, it was 10 years ago and I don’t remember the exact sequence of things.
This whole thread feels like an advert for FreeBSD. When you run
freebsd-update(to update the base system), it creates a new ZFS boot environment (snapshot of the root filesystem). No matter how badly it goes wrong, you can always revert to the old one (as long as you can connect to the console). After the update, you delete the old one. If it’s a major update, the new version will default to installing the the ABI compat components and so you can update packages later (and do the same kind of snapshot of /usr/local if you want to be able to roll those back). Doesn’t Ubuntu do something like this? I’d hate to do any kind of serious system maintenance without an undo button.That’s a really clever strategy and no, I’m not aware of any Linux distro that does something like that. Guix and Nix get to an even broader reversibility with a very different strategy: the packages are managed like an immutable hash data structure in a functional language. The packages are the keys, and the files are left in-place on disk as the package is changed on-disk by creating a new set of symlinks (pointers) into them. There’s an eventual GC process for long-unreferenced data.
As an aside, this is so obviously the right way to do package and configuration management that I evaluate them ~yearly for use on Lobsters and personal projects, but the documentation/maturity aren’t there yet and I’ve never seen a working install of a Rails apps as they have significant impedance with language-specific package managers/apps distributed as source.
honestly I have never had such problems running debian, so maybe just an ad for not running ubuntu on servers ?
The root snapshot reminded me of transactional-update and ABRoot, which are (as best as I can tell, I haven’t used them) both snapshot-based strategies for applying updates.
I’m curious why this isn’t standard practice even for distributions that you do trust to upgrade versions correctly: spin up a new VPS with the new distribution, copy over (hopefully a small amount of) state, switch DNS to point to the new VPS. What are the arguments against that approach?
The argument in our case is that ansbile has been brittle. Some of that is investing more time in cold-start provisioning, but some of that has been ansible making backwards-incompatible changes more often than we make any changes to our playbook. I never wanted to spend the (it turns out) ~5 hours getting that working.
Also, “switch DNS to the new VPS” has bitten us, the new one was assigned an IP that’s on many spam blacklists. I didn’t think to check and reroll until getting a clean one (this is apparently the best practice for DigitalOcean).
Yeah keeping up with ansible changes is tiresome. I found that it’s helpful to use a scratch server to debug the playbooks without any time pressure.
When I was doing email, we ended up making some effort to keep the IP addresses of our mail servers as stable as possible. Rather than updating the DNS we would just move the service IP address(es) from the old server to the new one - these were virtual service addresses, in addition to the per-instance addresses.
Dunno how easy it is to fling IP addresses around like that in a VPS provider: probably hard or impossible! In which case I might treat the scratch server as a practice dry run for an in-place upgrade of the live server.
On Linode, you can swap IP addresses between machines. On DigitalOcean, it looks like Reserved IP addresses fill a similar niche. It looks like that would be for a new IP address, not the current ones, though.
Yeah, I brought this up last night. I was planning to file an issue or something like that when the current crisis had passed so we can do an orderly transition to a new IP and not have as many problems next time.
I’m chewing on this one. I’m a bit irked that the product is basically paying the hosting provider to avoid dealing with problems caused by inadequate enforcement against bad behavior by other customers. The most uncharitable version of this is that the company is extorting me to avoid the expenses of acting responsibly. So my hobbyist sensibilities about the way things Ought To Be are clashing with my professional sensibilities about how cheap an improvement to prod uptime it’d be. Probably I’ll get over things and set it up soonish.
Reserved IPs are free when assigned to a droplet. They only bill you when you’re not using it.
Oh, I totally misunderstood the pricing and thought it was a flat amount per month. Thank you, I’ve added one to the web01 vps and I’ll transition DNS to it soon.
Interesting, thanks. So is it a case of live state that is difficult to copy to a new VM?
And thanks for the info on the IP address! I never realised that blacklisted IPs could be recycled to poor, innocent VM subscribers.
And I’ll add my vote of thanks for keeping Lobsters running!
Disadvantage: You temporarily double the resource usage, as both the old and the new server exist at the same time for a short duration. Another issue is DNS propagation which is not instantaneous.
Overall, it is still my preferred method because if anything goes wrong, I notice it early and can keep things running on the old server while I troubleshoot the new, upgraded one. For example, the outage described above wouldn’t have happened if this procedure was followed.
What sort of costs do we expect to incur in consequence? For example, Linode’s most expensive plan (dedicated 512 GB) is $7 per hour.
Would setting DNS TTL to 60 (seconds) sufficiently far in advance mitigate that disadvantage?
Considering the free nature of lobsters a $7/hr VM would quickly increase hosting costs even during cut overs if they things don’t go as planned.
Consumer ISPs often have their own idea of TTLs independent what the authoritative server tells them.
Lobsters finances (my wallet) is fine with this kind of expense. Running Lobsters would be a cheap hobby at twice the price. Just wanted to post a reminder as people occasionally express concern about hosting costs, and I prefer not to take donations.
The approach of running two VMs during cut-over doubles the cost only during the fraction of time that a whole-system upgrade is being made. In extreme cases that might be, what, 8 hours a year? That would add 0.1% to cost of the VM. I don’t see that it would “quickly increase VM costs”!
Perhaps the correct approach is to use some sort of reverse proxy but if that requires a VM of its own it would definitely add a lot more cost.
That depends very much on the server/plan. But yes, in the cloud, it usually costs peanuts. But if you have a beefy on-premise server, you can’t just double your hardware on a whim.
I don’t have enough experience with DNS management to know for sure. All I know is that this stuff keeps being cached at every step of the way and it makes life difficult sometimes. Unfortunately, it isn’t just a switch you can flip and it’s done.
That’s crazy. I’ve used “apt-get dist-upgrade” as the sole means of updating my Debian machines for 19 or 20 years now. Granted they’re desktops, and 90% of the time I’m going from testing to testing, but still, other than an Nvidia driver fubar once, it always works great for me.
Hard to argue with that advice, though.
The advantage of routine rebuild-from-scratch is that you get to practise parts of your disaster recovery process. When something surprising like a driver fubar happens, it does not happen on a production machine and it does not make a planned outage take much longer than expected.
I have also had no problems with Debian upgrades long-term. I’ve had two long-running Debian servers: one on physical hardware, installed in 2001 and dist-upgraded until I retired the machine in 2010, and one on a VPS, installed in 2013 and still running. Been very impressed on how it all just works. In the early years (2001-2006 especially) an upgrade would often break X11 and require me to futz with XF86Config, but that was eventually sorted out with better auto-detection. The underlying OS never had issues though.
I just upgraded from ubuntu 14.04 to 23.04 a month or so ago; I was pretty surprised how smoothly it went
I’m about 50/50 for
do-release-upgradeworking on my home servers. Failures are usually due to some weird configuration I did (hello, years of changing network configuration systems) or a third party package, I’m not mad about this, trying to support absolutely every possible Ubuntu system is impossible. But it is an issue for me.—Andrew S. Tenenbaum (author of Minix)
Ah, the days when the entire campus shared a T-1 connection and it was just easier for UCB to mail you a 9-track of BSD 4.2 than to try and download it.
Apparently he wrote this in 1981 [1]. It’s amazing how over 40 years later, the premise still holds true. There seems to be a fundamental principle at play that could not be broken with technological innovations. Makes me wonder if there is a physical law that limits the speed of transfer of information depending on the mass and energy that is used to transmit it.
[1] https://what-if.xkcd.com/31/
Reminds me of the AWS Snowmobile product for the transport of exabyte-scale datasets into and out of the AWS datacentre. Essentially a SAN on wheels.
We’re moving a ton of data around these days, for $REASONS. I have repeatedly pointed to this example when people respond to the station wagon analogy with “that’s just something you oldsters say…that doesn’t matter now”. Suddenly, a dim light goes on (an LED, not a light bulb).
As that xkcd says, “all of that data is coming from somewhere, and going somewhere.”
To really hammer home how quickly the price of storage is still shrinking, the article suggests about 900-1000 USD for a 1TB SSD, but you can buy a 1TB Western Digital Black NVMe on Amazon.com right now for $50.
Something beyond Shannon-Hartley theorem?
Perhaps? What’s the signal power of matter you take along? You just take
e = mc²? I haven’t thought this through in detail, I was just wondering as I wrote the comment.Thinking out loud here:
With digital data transfer you have to essentially take 1 electron at a time to the new destination.
With physical data transfer you can take trillions of electrons at a time to the new destination.
Your comment about e = mc² really does seem to touch this. We can only fling 1 electron near / at the speed of light at a time. So there’s a hard limit on how much data / second you can do. We can carry tons of electrons though at a time.
So we’re optimizing for “sending the least information that makes the most sense to us in the least time”, more or less. We don’t need to be sending each other 100GB videos, and we don’t. It’s much faster to send a 1GB video that is as recognizable as a 10GB video over the internet.It comes down to “we want to see something as fast as possible”. i.e. what most people say about this topic: optimizing for latency.I think it’s better to see things as “digital objects”, like for example, if you wanted the Ethereum blockchain. It’d be faster to get it on a flash drive than downloading it for most people, but it can’t be broken down further than its “useful unit” i.e. itself.
100% awesome thought :)
Someone could probably graph when these thresholds cross? :o
Most bulk data transfer these days is over fibre. With modern fibre, you can send multiple wavelengths simultaneously. The upper limit on this is one symbol per photon, though I’m not sure that there is a theoretical limit on the number of distinct symbols. Practical limits depend on the discrimination of the receiver and the flexibility of the bends (different wavelengths travel at different speeds because they retract differently so unless you have a perfectly straight fibre and send to photons down perfectly straight, there are limits on how closely you can space them).
Similarly, flash doesn’t store one bit per electron. Each cell holds a charge and that encodes one symbol. With MLC, there are typically 4-8 possible values per cell. In theory, you could build storage where you use different energy levels in electrons on a single atom to represent this. Quantum mechanics tells you both how many possible symbol values you can store and why it’s really hard to build something useful like this.
For both forms of transfer, the practical limitations of the specific technology have much more impact than the theoretical limits.
It’s also worth noting that most of these comparisons ignore the bus speed on the removable storage. My home Internet can download (but not upload, because my ISP is stupid) data faster than I can read from cheap flash drives. Commercial fibre is faster that most local storage unless you are able to read and write a lot in parallel.
Wow, didn’t know fibre could do that :o sick.
Yeah, the electron bit was just to simplify thinking around things, but you’re totally right. And good point about bus speed. I think the counter argument there is “you have the data at your disposal now”, so you don’t need to read it all off onto your primary storage device?
Last night before sleeping I basically concluded that if you can just add more wires (and I guess as you explained, wavelengths) then you can always beat the “practical carry limit” of physical data transfer… Except it seems you can always store more before you can transfer faster, at least that’s the trend right?
Innovation in tape storage density is a big factor.
I’d argue the principle is Latency Bandwidth relation.
You can absolutely start pushing enough bandwidth to beat the pidgeon over most ordinary wire. But the protocol involved will mean your latency will suffer and normal usage will be near impossible.
For example, you could write a protocol that spends a minute probing out the exact physical properties of the transfer wires between source and destination so it can model the wire and figure out how to blast through a 100 gigabit packet of data once. The receiver will be spending that hour much the same trying to probe out the wire. Once the blast is complete you’d have to remeasure, as it’s now another minute and the wire might have warmed up, changing properties. Plus the receiver now needs to process this gigabit blast and ACK the reception (even just hashing it to make sure it got there will take a moment with 100 gigabits of data). Retransmissions will cost more time. But ultimately this incredibly horrific latency buys you a lot of bandwidth on average, not a lot in the median.
On the flipside, you can not do that and get much lower average bandwidth but much more usable latency guarantees, plus a better median bandwidth.
I actually want simplicity.
This is why it’s so important to word such things carefully. Going by the article’s title, you either don’t exist or are no one. In essence, the post discusses away the existence of certain people and mindsets. It is not only intellectually disingenuous and wrong, it’s also rude. It’s a complete non-starter for a fruitful discussion.
Eh. I think reading beyond the title is underrated. The question is “what are you willing to sacrifice to achieve simplicity?”, which requires some explanation but does lead to the conclusion in the title if the answer is, in practice, essentially nothing.
OK, sure. “What are you willing to sacrifice to achieve simplicity?” would still have been a better title.
I did read the article and it doesn’t change the fact that the title is wrong and flamebait.
Why are you assuming that the only way to achieve simplicity is to sacrifice something? That is not true at all. A smaller simpler (but better) design can be more feature rush than a more complicated one.
I’m not, I’m simply stating what the article says.
But I do think almost everyone would agree that simplicity is better if there’s no trade-off necessary to achieve it.
Sadly, this type of wording in titles (also in videos) are the ones that get traction pretty much every where. It cannot allow for a middle ground in the point that (hopefully) the author is trying to move across.
There is no statement about everyone that can possibly be anything but hyperbole, other than the most basic statements of fact like “no one can survive without oxygen.”
No one should be offended by hyperbole.
(ba dum tss)
I work on safety-critical real-time stuff. We can only use simple software otherwise it becomes impossible to guarantee safety and timing. For many use cases a Linux kernel is too complex. A microprocessor is too complex.
Author here. Obviously my post’s title was inflammatory hyperbole, but to try to defend my main point for a minute:
How many things in your web development stack can you name — things that you actually like and consider to be a Good Thing — that you would be willing to sacrifice to achieve simplicity?
I can’t think of many for myself.
Performance; as long as the code isn’t doing things in a boneheaded way, it’s almost always fast enough.
Visual effects; I’m cheating here – this isn’t something I actually like, so removing it is pure upside. But it’s complexity that many many applications insist on, and removing it would be welcome.
Abstraction; code is often shorter and easier to explain when written with fewer, smaller abstractions. The abstractions may be elegant, but they aren’t worth it.
Resilience; the ability to handle faults gracefully is often fragile, getting it right is time consuming. It typically doesn’t pay it’s way. Detect the error, crash loudly, and move on. Be resilient in a small number of carefully chosen chokepoints instead of spreading error handling everywhere. Aiming for five nines of reliability isn’t worth it.
the list goes on.
Obviously my reply is inflammatory hyperbole….
Because we can’t be arsed to find another job, we let our manager convince us to add yet another baroque curlique to a product that should have been cast in concrete and declared “done” a decade ago….
…the only ethical thing to do actually is remove obsolete features and any remaining bugs.
But it’s still selling and unless we keep adding features we will be out competed by our competitors and creating a new product in a new category is too expensive and risky…
So day by day with a sick feeling in our stomachs we add yet another wart.
And our customers, being in a market for lemons, choose the glossiest and sparkliest lemon on the rack…..
…and so life gets shittier for everyone.
I already achieved simplicity. No frameworks and using basic text editors. I guess…
Hyperbole, yes. Inflammatory, no.
I would sacrifice the Lisp and ML families. I would sacrifice
sedandjq. I would sacrifice Nix.So, what are you willing to give up for it?
The point, as I take it, is that “simplicity”, like “performance” and other good things people often claim to want more of, is a tradeoff: if you want more “simplicity” or more “performance”, you often will have to give up at least some quantity of something else you care about to get it. And reframing in this way is not just splitting hairs, because it leads to more productive discussions. For example, instead of “why don’t developers care about performance” (unproductive) we can reframe to asking about tradeoffs between performance, which developers do care about, and other things that they also care about, and how to affect the choices being made.
Simply said
Most people don’t want simplicity, but the author just put “nobody” because it was simpler.
Do you own a dishwasher?
I do. And I installed it myself, which means I can claim with a straight face that dishwashers are pretty simple:
there’s a hookup for power. Electricity is used to heat water and run the controller which handles the user interface, sensors and timing commands. Finally, it runs a motor for a drain pump, possibly with a small grinder.
there’s a hookup for clean water input. Water is used to clean the dishes, and incidentally to spin the distributor arms.
there’s a hookup for grey water draining.
The primary differences between consumer dishwashers are the material of the central tub, the arrangement of the racks, and the arrangement, quantity and quality of the sound insulation. Most of the sound insulation is external and replaceable.
Once you understand it, it’s simple.
Apparently this is largely due unintended consequences to EU regulation on replacement parts for white goods. This meant that it’s not economically viable to produce dishwashers that are differentiated in any meaningful way unless you are certain to sell very large numbers. They all include almost all of the same components so that they can guarantee availability of replacements for the mandated period.
Having flooded the floor by not putting plumbers’ tape on when connecting up the water input, I can attest to the ‘once you understand it’ bit.
Surely that’s an intended consequence. Policymakers would have to be quite naive not to expect that outcome.
This is speculation, of course, please don’t take it too seriously. Your point stands, regardless.
I don’t think reducing competition or innovation was an intended consequence. Policy makers being naïve and not considering second-order effects is fairly normal in my experience.
Sorry for the delay in responding.
I of course don’t know what’s in anyone’s mind except my own, and you could well be right. I do think that regulatory capture is a transformation that most industries have gone through, and it would require a surprising level of naivete to not anticipate it. I am not an EU resident and I don’t really know any EU politicians, so I’m not really qualified to speculate I guess.
The API is simple, for you. The complexity is hidden behind the form of a commodity. Mining, manufacturing, transport, etc. It always seems like when programmers say they like when things are simple, they generally just mean “simple, for me”, and not simple, for thee. It seems like to me a proclamation for concerns to be handled by others.
All simplicity is subjective. if you disagree, I’m going to have to point out that you are made of billions of cells, each composed of millions of molecules, each built from up to a million atoms, each made of leptons and baryons and held together by forces that require conditions approximating the first minute of the universe even to observe reliably… so nothing is simple and the word has no objective meaning.
I know.
It’s important to note that PID controllers are a general concept and many embedded devices contain some implementations of PID controllers to control temperatures, timings, motor positions, etc. This is just one of many implementations of a PID controller. They make it sound like PID controllers specifically regulate temperature.
The intro of that article needs a link to Nicholas Minorsky who invented PID controllers for the automatic steering system of the warship the USS New Mexico. Rather a long way from a coffee machine!
When they installed new coffee machines at $WORK, I was astonished by what was exposed in the admin interface. The entire state machine for making different beverages is exposed, including temperature, flow rate, and (millisecond-precision) timing for each step. If you haven’t seen this, it’s well worth a look next time someone is servicing a machine. Apparently most of them have a sequence you can press that gives you a read-only view of the admin interface, so you can see exactly how your coffee is made, you just can’t change it.
I write software for commercial coffee machines (c++ and c) and us and most others have default codes for the service interface (try 1111, 2222, 12345, etc or lookup the service manual for your type online).
In my case the recipes are a sequence of steps, like, open this valve for x ms, run this grinder for y ms, move the brewer to position z, regulate flow).
I implemented PID code in the flow / pressure regulator for one of the brewer systems. With some special hardware it can auto correct to the requested pressure even if the coffee grind is too course or fine. It uses the flow data over time to give consistency, even if 6 months in your grinder has run in, theoretically giving a consistent beverage every time.
Go ahead and ask along if you want to know more.
(The main factory is in the Netherlands, but we also operate a Factory in Taylor, MI, for the American market, so you might even have seen a machine by us in the US)
Agreed. PID controllers are awesome. They’re an amazingly simple and general-purpose algorithm for any time you have a value and a target number, and you want to say “move the current value smoothly towards the target and keep it there even if the target moves around”.
Perhaps even more interesting is you can purposefully tune it to not be smooth, if that’s to the benefit of your system. The smoother it is, the longer it takes to get to the desired value. It’s often advantageous to quickly overshoot and then rapidly back off. Basically oscillate towards the intended value while smoothing out over time.
Your statement is generally true, I just wanted to add this bit that I find interesting.
they are incredibly common in industrial applications. they can regulate flow, heat, cooling, anything really. they have cascading pid controllers and many other. PI or PD are also common subsets.
Apple is well-known for its walled gardens, security theater and hostility towards general purpose computing for decades already. If you buy these devices and give Apple your money, you deserve what you get. You’re not in control when operating these devices. If that’s what you want, fine. If that’s not what you want, don’t buy Apple.
Walled garden, sure. But Apple has top-notch security and it is absolutely unfair to claim otherwise. Out of any platform available for the general populace, it is by far the safest device to own (in part due to its walled garden, sure) — from the hardware all the way to software. The only other device that is similarly well-protected is a Pixel with GrapheneOS.
I do like and use desktop linux as much as the next person, but we should all know that an npm install could literally do anything on your computer from encrypting your hard drive to installing a permanent keylogger, the only thing protected is installing a video driver, as per the old xkcd, and unfortunately there is still a certain mindset that everything is security theater (though it has improved over time, back in the days even Linus and the kernel were prone to this, until proven otherwise. Nowadays the problem is mostly userspace).
Citation needed. This is an often repeated claim for which there is zero evidence. When’s the last time a vanilla Debian installation was pwned by zero click exploits, as it happened countless times on iOS over the past couple of years? My falsifiable claim is that an ordinary machine with latest Debian stable is way more secure against digital attacks than any Apple deployment.
As for npm, makefiles, and so on: You are a developer, you know the risks: Vet the code or run it in a VM. It’s not rocket science (and yes, I know, VMs are not impervious, but they exponentially raise the difficulty of exploiting the device).
Don’t get fooled: Great security is compatible with user freedoms. The big corporations make people believe that these two things are mutually exclusive, but the opposite is true: A lack of user freedoms is a severe security risk by itself. A prime example are locked down Android devices bundled with uninstallable crapware.
Apple systems are safe when you define safety as the user not being in control of the device. To me, that is the opposite of safety - a device is safe when I can install and remove whatever I want. It is high time we stop allowing monopolists to define the term safety as they please.
That’s a fairly apples to Debian’s comparison. Last time I installed Debian, it didn’t include a GUI or a web browser. It did include the Linux kernel, which has had quite a lot of vulnerabilities. Just looking at the ones in the last month, there are a lot of use-after-free bugs in the USB stack. If the same bugs existed in XNU, they would be less easy to exploit because XNU uses kmalloc_type, which does type-pooled allocation, so none of these bugs would allow an attacker to get a type confusion bug and trigger exploits.
If you do get a root compromise on the Debian system, out of the box it doesn’t provide any secure boot, so it’s trivial for an attacker to modify the kernel binary image on disk and have persistent root access. In contrast, iOS and macOS have a full secure boot chain. iBoot uses a special compiler mode that provides memory safety (and a reasonable degree of type safety) for the code, whereas GRUB (the equivalent on a Debian install) had a memory safety bug last year that let you bypass secure boot and required all cloud providers to roll out updates that revoked the signing keys. Once you boot, the filesystem layout that Apple inherited from NeXT has a clear separation of system files and modifiable files, which makes it easy for Darwin-based systems to have an immutable base image, whose hash is part of the secure boot chain. This means that a compromise can’t modify any system files (though it can put alternative versions in your path). This makes cleaning up from a compromise much easier, because you have a large set of known-good things to use.
Beyond that, if you install a random app via apt-get, it will (with the exception of a very small number of things that come with SELinux policies) run with the full privileges of the user that runs it. If it has a vulnerability that an attacker can exploit, it can compromise all data owned by the user. In contrast, anything installed from the app store is sandboxed by default (with Apple’s version of the TrustedBSD MAC framework) and has no access to the filesystem except for its own private region and files that the user explicitly grants access to. Even apps installed from random sources on macOS often now come with similar policies. For example, UTM on Mac runs sandboxed and so a VM escape is limited to the files that the app has access to unless it also has a sandbox escape.
The difference is they’re actually trying to improve memory safety in the bootloader.
When was the last time a nation-state level threat actor wanted to attack vanilla Debian installations? For Android/iOS, there are companies which exist just to find these vulnerabilities to make exploits that are sold to nation-states, since cell phones are big targets for repressive regimes. Some examples are Hacking Team and NSO Group, Are they attacking Linux installs?
Most servers on this planet run some Linux flavor. Debian and Ubuntu (which is Debian based) are extremely widespread. They are certainly attacking Linux installs beyond Android, yes. Anyone who maintains servers with exposed ports on the internet knows that there’s constant probing and attacking going on.
You are comparing a server maintained by someone at least a bit knowledgeable about computers that usually run self-made, trusted programs and are otherwise completely sealed off from any kind of external environment, to a device that can be used by my grandma to install any kind of untrusted app, right next to all the family photos, credit card, etc.
It’s not hard to protect a single port of access, as opposed to a mobile’s huge surface area.
Server security can actually be incredibly difficult and the surface area is often larger than what is exposed on a personal machine. Also, servers can be and often are very high value targets.
Either way, the discussion has gone full circle. If we as a society let large corporations decide what code can and cannot be executed, we the people will experience a loss of rights and powers the likes of which has never been seen before in history. And precisely this is happening at an accelerating pace. As an example, it has now been proven beyond doubt that the US government pressures social media companies into censoring uncomfortable speech and political opposition (prime example: Hunter Biden laptop). These walled gardens are the perfect instrument to impose a wide variety of restrictions quickly and easily. By the time people realize that the security argument was a Trojan Horse, it’s too late.
There is no security reason why Apple’s or Google’s keys must be in the TPM and you cannot put yours in there to replace it. Is granny going to generate an asymmetric key pair to customize the system? No, she isn’t. Again: Freedom and security are not incompatible, they belong together.
It is remarkable how many people with technological expertise fall for this cheap PR. Pay 99$ to protect me… From myself? Preposterous. It should be called out for what it is: A grift, rent seeking, abuse, exploitation. And it should be boycotted.
Same thing with Web Environment Integrity. It has nothing to do with security and everything to do with control and power. No profit driven company cares about you being secure beyond the letter of the law, don’t make me laugh. One thing is certain: We will hear a lot more about remote attestation in the near future and it’s not going to be pretty.
because less people use gnu/linux on personal computers? but (this has happened)[https://www.vice.com/en/article/v7gd9b/facebook-helped-fbi-hack-child-predator-buster-hernandez] on debian-based systems as well
erm… npm works just fine on macs, as do any things it installs
I don’t understand your comment - my point is that Linux desktop has practically no security whatsoever: any program (e.g. npm) runs with the same privileges as your file browser, so the UNIX model is basically useless at that granularity. There are solutions, e.g. Android makes the whole thing secure built on the same base, but mainstream linux userspace barely cares about it.
OSX is a lot better, probably the best out of the 3, but ios is better.
I do love plantuml, but I wish there was a tool that could make diagrams from existing code instead of doing double bookkeeping…
Run your code through ChatGPT to write a prompt for DALL·E 2 which then generates your diagrams. Work smart, not hard ;^)
I think this is a joke but I use bing to generate dall-e 2 prompts - and claude for midjourney, I get better results than manually writing image prompts.
not for diagrams though.
I don’t know any off the top of my head for PlantUML specifically, but there are definitely tools that will generate UML diagrams from existing codebases (though I imagine you’re more interested in sequence diagrams and such)
I wrote this because I wanted to create diagrams periodically but I didn’t want to keep them up-to-date… https://github.com/glesica/dcdg.dart/
I have no clue what the author is looking for. Querying fonts with GTK (and all other GUI toolkits that use Pango, which is extremely widespread) is simple: https://docs.gtk.org/Pango/class.FontMap.html
Here is a real life Rust application that chooses a monospace font dynamically at runtime by intersecting a preference list with what’s available on the system: https://github.com/BenjaminRi/Sherlog/blob/0bd8ac46b1ec3c2628664cbdba3758bd5ecc2906/src/main.rs#L716
I don’t want to use pango.
Note that this is something that has already been addressed with the Printk Index. A good article can be found on LWN.net, of course.
It’s indeed weird the bug was WONTFIX’d, but maybe a patch for it will be welcome in the libvirt mailing list:
https://libvirt.org/submitting-patches.html
I’d do it myself but I don’t use libvirt much…
Yes, in Open Source, there’s no WONTFIX. There’s just “you have to do it yourself” ;-)
When I started doing Open Source contributions, I was surprised to see that pretty much all my PRs were accepted into various projects for which I wrote fixes and improvements. That was really motivational - I expected the reactions to be a lot more apathetic or even hostile. I’m sure such a patch would be welcome and eventually find its way into the upstream repository.
I miss the time when all the fonts were nice and crisp. It’s so hard to get modern systems to respect that without the fonts looking utterly terrible. These little pixel fonts have more artistry and craftsmanship in them than all those fancy vector graphics fonts we blur and anti-alias so carelessly.
In my experience, fonts look very sharp and crisp on a 27″ 4K monitor.
Are you arguing that I must quadruple the number of pixels on my screen due to software regressions?
無
Ironically, the practice of disabling anti-aliasing for CJK fonts / systems by default is very widespread due to the fact that these fonts are complex enough that you must make every pixel count and you can’t just vomit vector graphics all over the pixel grid. This is why you see Asian screenshots with crisp text even though ours is blurry.
Lucky you.
For those of us who care for monospace uses Terminus is available at sizes up to 32 pixels so it works well at 4K. Lucida Typewriter is another choice (up to 34px) but the commonly available version doesn’t have the zeroes dotted. Any others?
I use Dina Remastered (
DinaRemasterCollection.ttc).That only goes up to 12 pixel height right? I was hoping for more options that go up to at least 24 so that it can work well at higher DPI.
Yeah, it’s only up to 12px. Well, I guess 24px is just 12px upscaled x2, but you miss out on the opportunity to use those additional pixels for more well-rounded characters.
It’s not the blurriness that’s the problem for me; it’s the consistency. On a 4K screen I don’t notice, but anything less it bothers me if e.g. m and n have different weight due to different positioning relative to the pixel grid.
We need to tell people that literally anything they read or hear can be wrong. It’s high time that the majority of people gets this memo - in fact, it’s of civilizational importance. The “confident bullshitting” that ChatGPT exhibits is nothing compared to what reaches our ears all day every day from news and politicians. The damage caused by unfounded belief in what conmen say is untold - ChatGPT is a minor offender in the grand scheme of things. With a bit of luck, hallucinating AI will usher in a second age of enlightenment by forcing people to practice basic epistemological hygiene. Such a cultural development might be as impactful on human wellbeing as the practice of literal hygiene by washing hands.
I have spent a lot of time thinking about the nature of science and epistemological hygiene and wrote some posts about it over the years:
I believe that the ChatGPT issue is downstream from this. You can teach a computer to repeat what is commonly understood to be true, but you cannot teach it to tell the truth - and those two things are further apart than you might think. ChatGPT makes obvious an epistemological horror that has always been present - someone confidently telling you nonsense and you believing it. This problem is as old as language and the remedy is a set of intellectual tools to sort out incorrect information, taught to everyone. You can make ChatGPT a bit better, but that won’t solve the root problem. ChatGPT two millenia ago would have taught you the geocentric model.
ChatGPT for President. It’s made in the USA, so that’s good, but it won’t be eligible until 2057 because of the age requirement.
This seems a tu-quoque flavor of moral relativism. All peer reviewed studies are useless to me unless I have personally reviewed them myself? Some scientists have written some wrong things some of the time, therefore, all scientists and ChatGPT are equally unbelievable?
To the contrary: It would be epistemological relativism to blindly trust certain individuals or publications. Everything has to be read with doubt in mind. And every piece of information has to be investigated for internal contradictions or contradictions with known facts, that is the bare minimum to steel ourselves against incoherence and confusion, which are ready to enter our lives at every turn. If people learned and cultivated this skill, they would immediately see through ChatGPT’s fabrications, among other things.
I never claimed that. Peer reviewed studies are an important source of information. But just like every other source of information, they are by no means fully reliable. Our scientific institutions are plagued by issues like the replication crisis. Just recently, the Stanford president had to announce his resignation because it was uncovered that he was the principal author of papers with manipulated data. That’s just one of many examples of fraud in science. Just because they perform their work in historic buildings, add titles to their names, call their essays “papers” and publish in esteemed journals doesn’t necessarily mean that there is any truth in their work.
What an awfully uncharitable way to twist my words. I get the feeling that you were somehow emotionally put off by what I wrote, which lead you to construct straw-man arguments. My point is that people should vet and filter incoming information, no matter the source. Everyone who unquestioningly believes what ChatGPT writes was already intellectually deeply troubled before ChatGPT existed, for they let the garbage flow unfiltered into their mind and worldview. The problem is that we live in a society where people need to be repeatedly warned that not everything they read on the internet is true, a fact that should be painfully obvious even to the feeble minded.