If I was ever looking for an ‘out’ I think this was a fair trigger. A link to … a todo on a git forge with nothing of substance that hasn’t already been said a decade+ ago? Did I misspell phoronix.com? …
I’m not sure if it belongs here, but it’s not a TODO, it’s a WIP merge request currently with almost 2000 lines of new code that you can compile and try.
I’m surprised it works on Windows, because the kernel docs suggest it shouldn’t. DRM’d media is sent to the GPU as an encrypted stream, with the key securely exchanged between the GPU and the server. It’s decrypted as a special kind of texture and you can’t (at least in theory) copy that back, you can just composite it into frames that are sent to the display (the connection between the display and GPU is also end-to-end encrypted, though I believe this is completely broken).
My understanding of Widevine was that it required this trusted path to play HD content and would downgrade to SD if it didn’t exist.
No one is going to create bootleg copies of DRM-protected video one screenshotted still frame at a time — and even if they tried, they’d be capturing only the images, not the sound
If you have a path that goes from GPU texture back to the CPU, then you can feed this straight back into something that recompresses the video and save it. And I don’t know why you’d think this wouldn’t give you sound: secure path for the sound usually goes the same way, but most things also support sound via other paths because headphones typically don’t support the secure path. It’s trivial to write an Audio Unit for macOS that presents as an output device and writes audio to a file (several exist, I think there’s even an Apple-provided sample that does). That just leaves you having to synchronise the audio and video streams.
I’m pretty sure that what Gruber is describing is basically just “hardware acceleration is not being enabled on many Windows systems”, but because he has his own little narrative in his head he goes on about how somehow the Windows graphics stack must be less integrated. Windows is the primary platform for so much of this stuff!
I would discount this entire article’s technical contents and instead find some other source for finding out why this is the case.
Well it depends on the type of acceleration we’re speaking of. But I’ve tried forcing hardware acceleration on video decode and honestly you’d be surprised how much it failed and I did this on rather new hardware. It was actually shockingly unreliable. I’m fairly certain it’s significantly worse if you extend your view to older hardware and other vendors.
I’m also fairly sure, judging by people’s complaints, that throwing variable refresh rate, higher bit depths and hardware-accelerated scheduling in the mix has not resulted in neither flagship reliability or performance.
It can be the primary platform but this doesn’t mean it’s good or always does what it should or promises it’ll do.
I think it means: enabling the feature to screenshot DRM protected media would not by itself enable piracy, since people would not use screenshots to pirate media frame at a time.
What you are saying reads like “one technical implementation of allowing screenshots would enable piracy.” I trust that you’re probably right, but that doesn’t contradict the point that people would not use that UI affordance itself for piracy.
No one would use screenshots for piracy because all the DRM is already cracked. Every 4k Netflix, Disney, etc, show is already on piracy websites, and they’re not even re-encoded from the video output or anything, it’s straight up the original h264 or h265 video stream. Same with BluRays.
Yup, if you go through GitHub there are several reverse-engineered implementations of widevine, which just allow you to decrypt the video stream itself with no need to reencode. That then moves the hard part to getting the key - fairly easy to get the lower security ones since you can just root an Android device (and possibly even get it from Google’s official emulator? At least it supports playing widevine video!), the higher security ones are hardcoded into secure enclaves on the GPU/CPU/Video decoder though, but clearly people have found ways to extract them - those no-name TV streaming boxes don’t exactly have a good track record of security, so if I were to guess that’s where they’re getting the keys.
Still, no point blocking screenshots - pirates are already able to decrypt the video file itself which is way better than reencoding.
Those no-name TV streaming boxes usually use the vendor’s recommended way to do it, which is mostly secure, but it’s not super-unusual for provisioning data to be never deleted off the filesystem, even on big brand devices.
The bigger issue with the DRM ecosystem is that all it takes is for one secure enclave implementation to be cracked, and they have a near infinite series of keys to use. Do it on a popular device, and Google can’t revoke the entire series either.
Personally, I’m willing to bet the currently used L1 keys have come off Tegra based devices, since they have a compromised boot chain through the RCM exploit, as made famous by the Nintendo Switch.
I mean, things were overhyped and ridiculous, but can anyone say that the internet isn’t at the core of the economy?
1999-2006: Java
Still one of the most widely used languages, powering many multi-billion dollar companies.
2004-2007: Web 2.0
What we now just think of as “the web”.
2007-2010: The Cloud
Again, powering multi-billion dollar workloads, a major economic factor for top tech companies, massive innovation centers for new database technologies, etc.
2010-2015: Social media
Still massively important, much to everyone’s regret.
2012-2015: Internet of Things
This one is interesting. I don’t have anything myself but most people I know have a Smart TV (I don’t get it tbh, an HDMI cord and a laptop seems infinitely better)
2013-2015: Big Data
Still a thing, right? I mean, more than ever, probably.
2017-2021: Blockchain
Weirdly still a thing, but I see this as finally being relegated to what it was primarily good for (with regards to crypto) - crime.
2021-present: AI
Do we expect this “bubble” to “pop” like the others? If so, I expect AI to be a massive part of the industry in 20 years. No question, things ebb and flow, and some of that ebbing and flowing is extremely dramatic (dot com), but in all of these cases the technology has survived and in almost every case thrived.
All of these things produced real things that are still useful, more or less, but also were massively and absolutely overhyped. I’m looking at the level of the hype more than the level of the technology. Most of these things involved huge amounts of money being dumped into very dubious ventures, most of which has not been worth it,
and several of them absolutely involved a nearly-audible pop that destroyed companies.
Yeah, I was just reflecting on the terminology. I’d never really seen someone list out so many examples before and I was struck by how successful and pervasive these technologies are. It makes me think that bubble is not the right word other than perhaps in the case of dot com where there was a very dramatic, bursty implosion.
The typical S-shaped logistic curves of exponential processes seeking (and always eventually finding!) new limits. The hype is just the noise of accelerating money. If you were riding one of these up and then it sort of leveled off unexpectedly, you might experience that as a “pop”.
To me the distinguishing feature is the inflated expectations (such as NVidia’s stock price tripling within a year, despite them not really changing much as a company), followed by backlash and disillusionment (often social/cultural, such as few people wanting to associate with cryptobro’s, outside of their niche community). This is accompanied by vast amounts of investment money flooding into, and then out of, the portion of the industry in question both of which self-reinforce the swing-y tendency.
Not for everyone and also not cheap, but many projectors come with something like android on a compute stick that is just plugged into the hdmi port, so unplug and it’s dumb.
Yeah, I’ve been eying a projector myself for a while now, but my wife is concerned about we’d be able to make the space dark enough for the image to be visible.
I use a monitor with my console for video games, same with watching TV with others. I think the only reason this wouldn’t work is if people just don’t use laptops or don’t like having to plug in? Or something idk
This one is interesting. I don’t have anything myself but most people I know have a Smart TV (I don’t get it tbh, an HDMI cord and a laptop seems infinitely better)
It’s the UX. Being able to watch a video with just your very same TV remote or a mobile phone it’s much much better than plugging your laptop with an HDMI cord. The same reason why still dedicated video game consoles exist even if there are devices like smartphones or computers that are technically just better. Now almost all TVs sold are Smart TVs, but even before, many people (like me) liked to buy TV boxes and TV dongles.
And that’s taking into account that a person own a laptop, because the number of people that doesn’t use PCs outside of work is increasing.
I have a dumb TV with a smart dongle - a jailbroken FireStick running LineageOS TV. The UX is a lot better than I’d have connecting a laptop, generally. If I were sticking to my own media collection, the difference might not be that big, but e.g. Netflix limits the resolution available to browsers, especially on Linux, compared to the app.
massive innovation centers for new database technologies, etc.
Citation needed? So far I only know about then either “stealing” existing tech as a service. Usually in an inferior way to self housing, usually lagging behind.
The other thing is putting whatever in-house DB they had and making it available. That was a one time thing though and since they largely predate the cloud I think it doesn’t make sense to call it innovation.
Yet another thing is a classic strategy of the big companies which is taking small companies or university projects and turn them into products
So I’d argue that innovations do end up in the could (duh!) it’s rarely ever driving them.
Maybe the major thing being around things like virtualization and related things but even here I can’t think of a particular one. All that stuff seems to steem largely from Xen which again originated at University.
As for bubbles: one could also argue that dotcom also still exists?
But I agree that hypes is a better term.
I wonder how many of these are as large because of the (unjustified part of) hype they received. I mean promises that were never kept and expectations that were never met, but investments (learning Java, writing java, making things cloud ready, making things depend on cloud tech, building Blockchain Knowhow, investing into currencies, etc) are the reason why they are still so big.
See Java. You learn that language at almost every university. All the companies learned you can get cheap labor right from University. It’s not about the language but about the economy that built around it. The same it’s true for many other hypes.
I have a list of one-off experiments that I pick something from whenever it feels like productivity is dipping. This weekend feels like one of those. The pick this time is to make a renderer that can output some point cloud visualisations of process memory for a Looking Glass portrait display the predecessor to the less interesting ‘cloud AI connected bullshittery’ (https://www.youtube.com/watch?v=N24CqEvtXxk).
For the unfamiliar, those are basically just a hidpi screen with a lenticular lens sheet glued on top. They hide a json blob in the serial number exposed over EDID with the calibration data (sheet properties and alignment). Realtime rendering to it is ‘a bit’ weird in that you encode the view angles you want into a ‘quilt’ so it’s n*m renderpasses and then postprocess in a shader that samples the quilt with offsets from the calibration.
I had been imagining something like the ability to extend switch to operate on any aggregate type that has a hash-function definition available, but having a comparator generalizes that, and even supports some interesting use-cases like canonicalization-in-comparison (as @andrewrk asked about) as well as even fancier things like edit distance matching!
On a multi-user machine /tmp is shared. For downloads I would prefer some world-unreadable directory. Though I don’t even know what are the recommendations for things like that nowadays. On Linux there’s /run/user/<uid>, maybe this one? Does xdg have any dir for “user’s temp files”?
I know you’re replying to the last question of the parent comment, but, for downloads, if using XDG user directories, might as well use XDG_DOWNLOAD_DIR.
All my home systems are configured with NixOS and have users for everyone that might need to use them, so kind of yes.
But my PoV here is about doing things right w.r.t. security “and just in case”. If you have a system daemon that runs as a separate user, and it’s compromised, you don’t want it to snoop/modify your downloads etc.
I think that’s a bad place for downloads, since the spec says
Applications should use this directory for communication and synchronization purposes and should not place larger files in it, since it might reside in runtime memory and cannot necessarily be swapped out to disk.
Practically speaking, on many distros this directory is a tmpfs a few GBs or so in size, so you do actually run into the size limit, it’s not just a theoretical thing.
On my client machines, I create ~/tmp/ and use that as a location to download, unzip source bundles when I only need a single file, throw a clone of a project for a comparison. I have it set up as a ram disk with a 7 day retention so it works more or less like tmp. Additionally, I set the noexec bit on the directory which makes it a better place for stuff that I just want to dig through.
A huge amount of the stuff that I bring onto my box comes through the ~/tmp/ folder first, and after looking at it and deciding I need it for longer I will move things to a disk some where.
I realized at some point that there’s a lot of stuff I know I’d toss in 2 hours but I also didn’t want to put it in actual tmp because it’s shared and I don’t want to mess with permissions or deleting it later.
In the ‘low mental bandwidth’ category I’m continuing work on frontending the Himalaya e-mail tool, current state: https://0x0.st/8ZoL.gif and hope to be daily driving after a few days more. It’s the first step towards something hopefully more interesting, and that’s part of merging a combined timeline across multiple internet socials (IRC, mastodon, lobste.rs, fossil, discord, RSS …) into non-intrusive monitoring.
For more focused work I am reworking IPC synch to (finally) drop the use of named primitives. The long holdout (outside of my general laziness on the topic) has been POSIX semaphores instead of FUTEXes thanks to the general footgun armory of all things system OSX. I have been hesitant to use the __ulock_wait/__ulock_wake facility for long enough that it is about time to take the plunge.
An anecdote from those times was that students had to copy the client from a public SMB share and run locally, while the teachers and admins ran it from the share directly with write permissions. The effect was that any messages or attachments they opened would write into temp files on that share yet those got world read permissions. So you could just scan/scrape that folder and get all the dish on who was flirting with whom or uncensored opinions on various students, password change responses, exams, …
Finishing a note on untested.sonnet.io (no strong schedule this time, just writing for fun), then adding one small UI fix to the site. Done, in this exact order so I don’t get distracted by code.
Then messing about in the kitchen, I learned that Portuguese grelos is the same as the Neapolitan friarielli and there’s a guy on my street seeking literal heaps of it.
Playing Disco Elysium while the food is getting ready. Becoming a magnesium based life form.
my favourite remains a shitpost of theirs, the “cryptography tier list” - a good use of AI voice cloning. I wish presidential discourse was at that level.
I hate saying this, but I’d love a TL;DR summary of these articles because they’re so lengthy. I’ve seen the others in this series posted here, and I still disagree with a lot of the premises. I don’t want clickable URLs in my shell buffer, I don’t think dragging a file onto the shell window should copy the file there, etc. There are just better ways of doing most of those things (IMO).
I don’t really get the focus on terminal emulation and TTYs. Nobody’s forcing you to use that stuff. There are a lot of ways to do “terminal stuff” outside of a terminal, with or without a CLI and TUI.
The obvious way would be to use Vim or Emacs and do the work there. Emacs has a TUI and can (or can be made to) do all of the things the author is asking for, and the underlying details of TTY protocols and terminal emulation is entirely hidden away.
I’m an emacs user, so that’s the only one I have much direct experience with.
I’d imagine most editors can do it. I’ve seen a half-dozen “magit for $something” articles posted here, so probably those tools. VSCode has a shell/terminal thing.
It doesn’t mean it’s correct, though. I like the the separation of concerns, and so an editor should be an editor. What people are doing in the e.g. Neovim community is insane to me (package managers, many Lua rewrites of tools you can use in the CLI, etc.).
I think the evolution needs to come for « CLI replacements », and so far besides Cat9 I really haven’t see anything promising (besides emacs, but it comes with its own issues).
We likely consume, and expect to consume, content on the Internet in very different ways. I personally don’t deal with short-form whatevers at all. If it’s not worth at least an hour with a printout in the arm chair I simply stay away.
You are forced to use terminal emulation and ttys every step of the way (assuming Linux/BSD). The only one that successfully paved another way is Android and it worked rather well for them, and even then it pops up here and there. I know, I worked on it. The first thing your desktop has to do in order to get something in the screen is a ridiculous dance to tell the TTY to please stop doing its thing, and even then it’s there in ways that completely undoes the security model they try to implement.
That something is ‘hidden away’ doesn’t mean that it doesn’t influence every layer above it. Emacs pays through the nose to get its TUI to work. There’s a ridiculous amount of code and workarounds to present its rather modest UI.
A tangent perhaps but as a former Emacs user and as former Vim user (neither are even close to doing the things I ask for) I find it fairly bizarre to use a text editor to do shell work and vice versa. Just as I avoid trying to use a tooth brush to clean my toilet or a toilet brush to clean my teeth. They can get the job done, sortof, but at a price and with some risks.
I’m going to disagree with your last paragraph and say it’s quite natural to use a text editor to do shell work.
Most “shell work” involves navigating around, searching through, and otherwise working with text, which is exactly the toolset built into text editors. cat, grep, sed, and awk are command line tools implementing text editor functionality in a clunky (clunky for interactive use) pipeline/streaming fashion. Likewise, ls, find, cd, and other filesystem navigation is also basic text editor functionality. They all spit out text, and what better way to view and work with it than a text editor? Maybe I’ve had too much Emacs koolaid, though.
For the rest of it, I don’t care much. On a day-to-day basis I never have problems related to TTYs, and with a few (easily fixed) exceptions in the past, it’s been that way for a long time.
With that logic you can use a hex editor for everything, it’s all “just” bytes no? representations matter, interventions matter. Trivialities aside, they operate on different mechanisms and assumptions. Text editing is buffers with some intended persistence and at rather small buffer sizes. You can expect to seek and build features assuming that works; the backing store changing is an exception, not the default and so on.
The cli shell is stream processing. Its data sources are hierarchies of short lived processes that might expect something back from you to do their job and can spit out infinite amounts of data. The world can change beneath its feet between every command. To do that it gets to go through all the ugly legacy – signals, sessions, buffer bloat, line blocking and with that, serial communication.
This brings us back to never having problems; ‘works for me’-isms is just dismissed with ‘doesn’t work for me’-isms and doesn’t get you anywhere. It’s systems engineering, one doesn’t get to think like that. You don’t have to dig far to find people with serious problems with the TTY and a few vulnerabilities a year. First(?) Flatpack sandbox escape? the tty again.
Whenever they try to fix something to make embedded development bearable, you commit the Linus taboo of breaking userspace because your stack relies on hidden assumptions that all go back to the emulating the fantasy computer. To edit text. It’s holding everyone back.
Just the other week I had a coffee spill. It fried my mouse and worse, I had to make a new cup of coffee. I grabbed another rodent from the box of spare input devices. I plugged it in. Now my keyboard stopped working. I unplugged the mouse again. The kernel crashed and I lost a lot of work. Angry reboot. Plug back in. Now there’s half a second delay between keypresses every 10 seconds. 2 minutes later the machine froze up. It turned out the USB mouse exposed itself as serial rather than USB human-interface. This meant some /dev/input/eventN gets routed through the tty layer again. The display server ran the device node non-blocking, but that only goes skin deep. You still have ‘the global’ lock around in tty and a handful of mutexes on top of that. Race condition into memory corruption and panic. Then race condition into deadlock.
The point you missed is that a text editor has tools to work with text, while hex editors typically do not.
I find working inside of emacs generally easier than in the terminal because it has great tools for working with buffers of text, which is what most shell commands produce. If there were a hex editor that made it even easier, then I would indeed use it instead.
This is just moving from the unix-style cli to the old win32-style cli. Except Microsoft gave up and ended up embracing ANSI escapes and CSIs (some time after WSL landed) to support the ever-growing corpus of apps specifically targeting terminal emulators.
In the Win32 console model, apps emitted only text to stdout and used the console handle to do everything else from setting attributes to switching between the full screen buffers. And the open source world made fun of them for it.
And the open source world made fun of them for it.
I wouldn’t do that, it isn’t like the unix terminal design is exactly ideal either.
I don’t think the unix design is as offensive on the output side as the input though; if you’re just writing a program that wants to output some colorful text, it isn’t all bad. Just remember to check isatty and the command line flag that overrides that before adding the color sequences!
On the input side though, yikes, what a pile of disgusting hacks. You have to do a timeout check on the esc char! And woe be upon you if you need to query the terminal for something, since now that’s both input and output sequences! Worth noting a lot of the newer extended terminal emulators are introducing new features to change this stuff, but they’re tied by the need to be compatible with the large existing ecosystem of applications.
There’s also the separate question of api vs underling data too. In the underlying data, one stream does make synchronization and redirection and other things easier. But then in the api you might want it separate anyway, to make supporting --color=auto and --color=off easier. Stripping escape sequences from the output isn’t awful though, so you could just always write them in your code and have the library function just does that.
Nevertheless, I’m not a fan of making fun of designs… there’s often fair reasons for differences that you can learn from.
If you generalise to that level, well, duh. The argument that in-band signalling UI is harmful in the same way smashing strings together versus prepared statements is not a difficult one.
Microsoft didn’t “give up” – it was a no brainer for them to throw a few idle headcounts on writing an emulator for their WSL trajectory and getting more developers into VScode. They’re otherwise busy adding AI keys and ads in the UI so perhaps their sensibilities is not something to use as ‘appeal to authority’ or inspiration.
I am intimately familiar with the inner workings of the different console layers in the MS ecosystem and their reasons for being – from the days of conio.h, int 21h, mode con codepage prepare and onwards. Their powershell model is much closer in comparison but the power they had there came specifically from them having something of a functional IPC system to work with. Linux/BSD isn’t so lucky.
It was a junior being thrown into the deep end of the pool (at his request) and had zero experience with the API (unsurprising) but also with python bindings. It’s not been kept up to date as my personal policy with python is that if something is written in it I don’t use it unless at gunpoint.
We used it for a monitoring system for a turnkey distributed ‘from breakpoint in vscode / .qcow2 upload in webUI’ → ‘analysis / harness generation’ → distributed across cluster → crash collection into triaging → UI callback’.
What someone should do though, is to take the -server end of this, embed into whatever terminal emulator is in fashion and suddenly there’s a seamless transition path to bootstrapping in other desktop environments outside of Arcan.
This is IMO a great example of why process isolation is the only path to confidentiality on modern CPUs. Site- (or origin-) process isolation has been and continues to be the durable path forward here.
It’s even directly mentioned: “our proof-of-concept opens Proton Mail’s inbox in a new window, but uses the same process to render the inbox” (emphasis mine).
Also note: the CPU is not the problem here IMO, it’s doing exactly its job.
Considerations for Safari. We emphasize the importance of site isolation [55], a mechanism preventing webpages of different domains from sharing rendering processes. Site isolation is already present in Chrome and Firefox [42, 55], preventing sensitive information from other webpages from being allocated in the attacker’s address space. While its implementation is an ongoing effort by Apple [3, 4], site isolation is not currently on production releases of Safari. On the contrary, we also reflect on [Webkit’s memory allocator] libpas’s heap layout from Section 6.3, allowing sites to not only share processes, but also heaps. Partitioning JavaScript heaps by at least several memory pages per-webpage would prevent JavaScript strings from the target webpage from being allocated within the 255-byte reach of the LAP.
i.e. an actual fix for the xz backdoor was to move code into different processes; to prevent Jia Tan code from sharing an address space with sshd code
(This was about a backdoor, not information leakage, but I think it’s still relevant – process isolation mitigates many different attacks)
I think a main issue is that process separation can introduce some nasty programming problems, making life harder for the developer (not to mention portability problems)
But it makes life better for the user, which should be the goal
So hopefully we will continue to find ways to make it easier, to have the best of both worlds
I have been isolating parser -> render now since about 2008. After Spectre I went ‘oh this is not enough’ and started using 1 task, 1 device then re-image and reboot.
A previous side-hustle (2015 or so) was a plugin that did that to ‘open attachment’ at a law firm that had more phishing attempts than actual e-mails. A pool of runner VMs in their cluster, each attachment got a fresh image. The weak link in the chain was the dependency to printers still but that at least was monitored heavily anyway.
Figure 1: A diverse collection of keys (generated with AI)
Fuck it. The image is unnecessary, ugly and my reading experience is made worse by its presence. I’ll follow the recommendations from the cryptography blog with the beautiful furry art instead.
That key image literally broke my pattern recognition (as many GenAI images do) do this point of my eyes unable to focus on the actual page and had to print to text and read it that way. Only other time I have had such a strong reaction was when playing around with an eyetracker and using the gazepoint to warp the mouse cursor. A really bad idea and I think I know how a cat feels around laser pointers now.
That aside, the recommendations are solid and the writeup good (as expected from trailofbits).
I personally just glazed over it to continue reading.
Didn’t notice it till I scrolled up again because I saw “figure 2” and thought..
“Hmm what was “figure 1” I don’t remember…oh! It wasn’t important.. Oh that’s AI generated? Okay”
IMO it’s a pointless addition, whether AI generated or not.
I don’t think we should blame (only) the authors, but also the ecosystem, because at the moment every “social site” practically requires an image to be shown as the “snapshot” of a shared link, thus the following HTML meta-tag which happens to use the same image as the first one in the article:
What happens if you don’t include such an image for each and every page of your site? Technically nothing, the social site just puts a blank image, a placeholder, or a scaled-up variant of the site’s favicon. However in practical terms, I bet the link is downgraded by “the algorithm”.
Thus, just ignore the blurb images, as I do almost every other site out there, as they are a requirement in today’s web-conomy.
I get you, but it could also be any image, any image at all. I often just use vacation photos. There is no requirement that it be a distracting bad AI image. The flippant use of an AI image is a strong quality indicator that implies the rest of the post will be garbage too. Better for the authors to have put in some effort.
I get you, but it could also be any image, any image at all. I often just use vacation photos.
Another piece of the web-conomy machine you are dismissing is “copyright” and “licensing”. Sure, for you blog, you can use your own vacation photos, because it’s not like you’re going to sue yourself for using them; but this is a company’s blog, written by a company employee (or contractor as it’s most often the case), thus he can’t use photos from his own vacation, because then he also has to transfer copyright, or at least license that image for the company to use, thus lawyers. Another option is to use some stock photo, but that means money, and it’s not like a bland stock-photo image is better than a LLM regurcitation. Thus, the cheapest solution is to make the LLM spit out some blurb image that has no copyright, thus no legal entanglements.
The flippant use of an AI image is a strong quality indicator that implies the rest of the post will be garbage too.
I can assure you that the post text is not garbage. It has a few good points that are expressed in plain words, instead of the crypto-highfalutin language other cryptographers use in their blog posts.
No yeah, I agree, the article is fine, it’s just the use of AI slop ruined any first impression, and I can see why many people didn’t bother to read any further. It probably did seem cheap to the authors at the time, and that’s a shame.
most CMSes allow to configure a site-wide fallback for og:image if the article does not have one. It can be a properly sized logo, or a photo of the author. If Wordpress does not support this, I guess there’s a plugin to implement it.
less than one minute of searching through unsplash surfaced this image of key blanks among many others. It is nicer, free to use (even on commercial websites) and is not built on stolen work.
This is super interesting but I’m having a hard time understanding the full extent of the post. Is there a project implementation somewhere?
I’ve been thinking strongly along the same lines for a while now. We need to break free of terminal emulation and in-band signaling for text based interfaces. I’d be super curious to read more about efforts in this area.
It’s distilled experience from 10 years of slowly sifting through the ~130kloc of linux-tty; 200kloc of ncurses; 70kloc of GNU readline; 70kloc of tmux; 150kloc of openssh + a handful of emulators (excluding all other display server things) – then abstracting and implementing. All because a former colleague dared me to. Now I can claim my beer. That’s to say there is quite a few moments of ‘oh no’ that change your mental model of things that’s hard to convey without some serious code-fu.
There’s quite a few people with insight and experience by now on the Discord (dang kids rejecting IRC).
I recall someone reproducing ACME in Rust as well, but I can’t seem to find the link, it was very “on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.”
Yeah. Telling users “if you want message history or to be able to receive DMs while not logged in, just run your own bouncer on a server somewhere” is ridiculous.
I’m shocked that Arcan uses Discord. I thought free software projects were the last bastion against it and of all the projects I would never have expected Arcan to capitulate…
Understand the enemy and get behind enemy lines, talk to the locals and gather intelligence. I don’t plan on staying there for longer than necessary but the alternative is early in proof of concept. In the current slopjob of an Internet I see Discord as one of the more dangerous of accidents (accident in the sense of its original scope of coordination/paratext around gaming).
The quality of information and community if you know which ‘fake BBS’ (sell the appearance of distributed, ‘click to spawn community infrastructure’) to look for is something I’ve not seen elsewhere, not on IRC, not on Usenet, not in mailing lists and so on. Their owners are getting a very good training corpus when other wells are poisoned.
We need to break free of terminal emulation and in-band signaling for text based interfaces. I’d be super curious to read more about efforts in this area.
A case worth looking at is the Windows console api…. which moved in the opposite direction. They had something free of that stuff and went toward terminal emulation, mostly in the name of compatibility with the existing ecosystem of terminal stuff. That’s a strong pull, and hard to imagine adoption of something without two way compatibility there, which does limit your design space somewhat.
But the Windows api - much maligned by some (and tbh its default user interface is kinda meh), but i like it - is more to the PC hardware text mode what xterm is to the vt100. You see the design influence and there’s some emulated compatibility, but also new concepts layered on top that weren’t in the original hardware (of if it was, i didn’t know how to use it lol).
In the hardware text mode, the screen buffer was composed of 2 bytes per character cell. One was the character code, which mapped to the font, the other an “attributes” byte, which was packed bits of foreground color, background color, and blink. Being an in-memory buffer, of course, you can poke those bytes in any random order or do block copies or whatever, but the print apis would generally just write the characters with a default attribute byte. Input was provided entirely separately since they’re separate hardware.
In the win32 console, they expanded on this: the character and attribute thing there is 16 bit, but same idea, they associated a current attribute with the screen buffer then provided functions to write text separate from change attribute, generally abstracting it into functions rather than an exposed memory block (though you can ask it to copy the memory block in and out for you - see ReadConsoleOutput), but you can see the heritage pretty clearly. So to write out some colored text, you first SetConsoleTextAttribute(handle, red); then WriteConsole(handle, "text");. This drives people coming from linux nuts because they so badly want to just printf("\033[44mtext\033[49m")… but there’s a lot to like about separating those things.
The win32 console also lets you allocate entirely separate screen buffers, write to them in the background, then swap it out to be displayed. Doing this with a unix terminal is…. non-trivial, and this is one of the functions they just plain left behind in their transition to unix style. But it is very useful, letting a program work in relative isolation. (In unix terminals, they usually set the magic code for “alternate screen”, vim does this for example, but there’s just one alternate screen, you can’t have multiple that you swap around.)
Input also works differently than unix: there’s two modes. ReadConsoleInput gives you an array of structured input events. Key down, key up, mouse moved, window resized, etc., all different tagged structs. MUCH nicer than reading random sequences on a timer and decoding them according to random environment variables! You can loop on these like any other event loop and find joy. Or you can ReadConsole, which just picks the characters out of it.
ReadConsole is interesting too because when you enable line buffering, the system provides a line editor for the user - arrow keys work, backspace works, there’s programmable history (Get/SetConsoleHistoryInfo), there’s even a built-in tab completion hook! (I could never find decent documentation of this so i wrote something myself on stack overflow then expanded on my blog https://dpldocs.info/this-week-in-d/Blog.Posted_2019_11_25.html#tab-completion i got an email once from a guy who said he worked at Microsoft who said that was the best doc he could find on it too lol). In theory, you can do some of this on a linux terminal too - cooked mode read line is buffered but it is more under the kernel’s control than either the terminal or the application, hence why most everyone uses gnu getline (or one of its many similar alternatives) instead when they want a richer or customizable user experience, but the Windows API is reasonably rich and customizable out of the box.
All this said… look up the docs to those functions, and every one of them has a deprecation warning, saying use the virtual terminal sequences instead. Sigh, feel like a step backward. But I understand why they did it given the ecosystem pressure.
Still, the Windows API is worth studying as another way to break free of these terminals, and from what I’ve seen of Arcan, in this post and others, looks like they’ve studied it too.
In the win32 console, they expanded on this: the character and attribute thing there is 16 bit, but same idea, they associated a current attribute with the screen buffer then provided functions to write text separate from change attribute, generally abstracting it into functions rather than an exposed memory block.
Back in the mid-90s, I wrote my own “terminal API” for the PC text screen (my own TUI as it were). You could define a region of the screen (default region—the entire screen) and do things like ScrollUp(region,n) (to scroll the region up n lines) or ScrollLeft(region,y) (to scroll the region left y lines) or even SetFGColor(region,c). At the time, I didn’t support separate screen buffers, but I think it could have been added with no issue.
And I liked this API, but how well would it lend itself to a “remote screen”, say, across a serial or network port? Some form of protocol would have to be defined to separate the commands from the data. And as for supporting existing physical terminals, there would have to have been a translation layer because I don’t know of any physical terminal supporting left/right scrolling (although I would love to learn othewise).
MUCH nicer than reading random sequences on a timer and decoding them according to random environment variables!
When Unix was being developed, there were dozens of different physical terminals available, all working over serial ports (the USB of the day). But the command sets and features vary wildly. The ADM-3A was a very popular (read: cheap) terminal, but it’s capabilities were extremely limited (you could move the cursor, clear the screen, and scroll the entire screen up—that’s about it). But the VT-100 was way more capable, and later models even more so. Unix solved this with a library, using an environment variable to select the terminal type and managing the differences so you didn’t have to.
The only reason a timer is required is because of the ESC key—you need to determine if an ESC character is a key press, or part of a control sequence. It would have been nice had terminals sent a control sequence basically meaning “ESC was pressed” then all the timing would have been avoided, but alas, we have to use the terminals we got, not the terminals we want.
Indeed… and they actually still kinda do, but you can’t use TERM since everybody just claims to be xterm to avoid being treated like a dumb featureless thing by ncurses. Reminds me of User-Agent: Mozilla 5.0 (compatible; MSIE….) in the browser world. Environment variables are kinda meh for this anyway (whether TERM or the TERMINFO ones) since you might move between terminals and need to adjust too; screen, tmux, etc. of course can offer a layer of stability but it’d be nice sometimes to just send down a “relocated” event with new information. (I guess in theory you could hot-plug hardware terminals too, but in practice that’d send a SIGHUP surely anyway.)
It would have been nice had terminals sent a control sequence basically meaning “ESC was pressed” then all the timing would have been avoided, but alas, we have to use the terminals we got, not the terminals we want.
Aye. Even now, 50 years later, we’re stuck with the terminals we got. There’s some newer ones that will send the CSI character instead exclusively, but you need to detect and swap and probably still check the esc char for compatibility anyway so, partial success at best.
I hope some day something like Arcan can manage to finally deliver a terminal we want, but it is definitely an uphill battle.
The only reason a timer is required is because of the ESC key—you need to determine if an ESC character is a key press, or part of a control sequence. It would have been nice had terminals sent a control sequence basically meaning “ESC was pressed” then all the timing would have been avoided, but alas, we have to use the terminals we got, not the terminals we want.
You have other timing channels on the input end - in the sense of the instruction decoder needing to do something when there is nothing in the immediate read buffer after an ESC, OSC, DSC and so on – even though that can happen naturally from a network stall / dropped packet retransmission / … It ends up in a heuristic in the emulator and tty implementation: do you consider blocking until you get more even though that would interfere with the software you are hosting?. You don’t notice it as clearly as with the ESC keypress (we are quite sensitive to jitter and judder). Networks are much faster these days and ncurses et al. tried to smooth it over by using a bandwidth estimator against baudrate so they don’t emit outputs until it’s unlikely that a sequence would get fragmented. I’ve run into plenty of cases where inputs and commands gets misinterpreted one way or another on slow serial links trying to provide a TUI (network routers, debug interfaces).
On the output end you have other timing breaks, an easy case is looking at how an emulator is handling “find /”. You can’t block until it’s completed or people complain you’re unresponsive. If you align to some artificial vsync from outer display system others will complain that you are ‘slow’ when they benchmark emulator a vs emulator b by timing how long a command will take (yet its actually the heuristic being measured). At the same time you don’t know what is actually running because a shell is in the way, so another set of heuristics depending on if you are in altscreen versus line mode and so on. This breaks when you nest terminal emulators (as the inner one becomes altscreen to the outer one even though the actual command is in line mode). Compare running find ‘raw’ and within tmux.
On the sideband end you have others in the signal propagation mess, obvious one is SIGWINCH on drag resize; be too quick to process and propagate resize events and contents will tear and corrupt endlessly. Be too slow and it will feel laggy. This is one you try to mask by padding with background colour and retaining ‘old’ contents hoping the user won’t notice as well as have rate limit or bin on something like the X11 XA_WM_SIZE_HINT width_inc and height_inc matched to cell sizes…
No, but you can represent the associated emphasis with color (and on Windows, you can query the background color to adapt accordingly!). This is done often on many linux terminal emulators too. For example, mine does underline, but ignores italic and treats bold as just a color modifier. xterm does bold, but treats underline and italic i think as a color change (that might be configurable im not sure).
Until Vista, you could hit alt+enter and bring a console full screen, using the actual hardware text mode too.
Of course, extending it to support these things would not be difficult (in theory), there are some bits available lol. (Tho if I was redoing it, I’d probably not keep the char/attributes packed like that, and instead put the attributes as a separate set of overlay ranges, more like a rich text component’s innards.)
Depending on when the API was defined, Unicode might not have been a thing at all (Windows 1.0 was released in 1985 after all). And it wasn’t until the mid-to-late 90s that you got sufficiently high resolution with more than 256 colors on PCs (yes, VGA, available in 1987, supported a 256-color mode, but that was only at 320x200). It was also probably developed to handle both text mode and graphics, thus some of the limitations due to text mode.
The reason I commented on the limitations is that it is so obviously constricted by the time it was designed, without leaving any space to grow.
It’s sad that the most prominent alternative to the terminal
interface had such lack of foresight, and the supposedly worse technology has been able to incrementally adapt from 1 bit to 4 bit to 8 bit and then 24 bit colour, from ASCII to Latin-1 to unicode with bidi and emoji.
There’s something weird going on here that I’m struggling to articulate. I think there’s an uncanny valley between the strict limitations of a terminal and the wide open possibilities of a GUI (or the web), where there isn’t a happy medium that’s as easy to program as a terminal but more graphical or more reactive or something.
I admire people who are trying to find or make that happy medium, but I fear the opposing pulls to the terminal or the web are so strong that building a middle ground is much harder than we might hope.
There’s another weird thing going on here too … stuck with sub-optimal solution because of backwards compatible (probably for things you don’t really want to concern yourself with) and being forced on the upgrade treadmill (because you want the feature—it’s new! It’s fresh! It’s the new shiny!).
Another issue is that it’s easier to combine command line programs (shell scripting) than it is to combine GUI programs. While there have been attempts to script GUIs (Amiga with REXX, Apple with AppleScript) none have really caught on. I have a feeling it’s because most people aren’t computer literate (computerate?), nor are they taught (in my opinion) true computer literacy (which is “computers can do repetitive tasks”) and GUIs are not really geared for it either.
The API behind the appls of Arcan is a long running attempt of finding such a thing, and I’m fairly content with the capabilities by now, but it took a lot of dogfooding. In that sense getting rid of curses and terminal emulation was a side-quest.
What surprises me more is the very existence of Electron. There ought to be a package format for the ‘app’ part and chrome itself capable of launching it. Having a fork that drags behind for such a massive and evolving codebase just hurts packaging and increases the value of n-day exploits.
Those were “Chrome Apps” which almost nobody used. I believe that was in part because Google kinda led people to believe they had to distribute those apps through the Chrome web store. That wasn’t really technically true but the developer documentation all assumed it.
One of Electron’s big draws is making it cheap to write a single app that supports the two most popular desktop operating systems, Windows and macOS. So I’m not at all surprised that no such package format exists, given that the format’s author would have to convince Microsoft and Apple to support the format.
macOS’s closest thing to a package manager is the Mac App Store. It has a lot of limitations, including not supporting third-party registries. So Apple is far from being able to support a specific package format.
Microsoft supporting a new format would be more plausible. Windows finally added the Windows Package Manager to the OS in 2020, seven years after the release of Electron.
Microsoft basically does support the web app stuff. Their WebView2 is an auto-updating embeddable chromium/edge for applications to use and you can declare your intention to use it with a little MS-provided dll or in the packager of the IDE (I know that’s vague, I’m not an IDE users, but I’ve seen the checkboxes in there before).
I know this isn’t exactly what you have in mind, since you still package an exe and dll, but it really isn’t awful.
So I’m not at all surprised that no such package format exists, given that the format’s author would have to convince Microsoft and Apple to support the format.
I don’t think this is true. It would be better that way, sure, but you can do it in userland. Instead of embedding Chromium the Electron app would embed a thin, infrequently updated runtime management layer that would manage the Chromium runtime in some common directory that every app used.
Lots of flaws with this approach obviously, many that would be solved by the system managing it, but I think most if not all of them are solvable decently satisfactorily.
Having Microsoft support it would be in their interests as they can then route it from App Store through Edge. That is enough surface area for it to be relevant.
Can someone ELI5 why I should care about esims? What actual problems of actual users do they address?
My (perhaps uninformed) take was simply that physical sims were a victim of Apple’s quest to have the iphone look as much as possible like a solid seamless block of glass and aluminum with nary a button nor port to be seen. Would I actually benefit in any way from using an esim in a device with a physical sim slot?
When traveling, especially to remote locations, obtaining an eSIM before the trip can be more convenient than trying to obtain a card locally, again especially if you want a specific plan (vs whatever a local shop might stock). Some plans are only available as eSIMs, so if you want one of these being able to do it in any device is useful. Very few phones can handle more than 2 physical SIMs and swapping them is kind of annoying, many phones can store more eSIMs - how much you need that again depends on your travel and usage patterns, for me it would’ve been quite nice before free EU roaming became standard, nowadays it doesn’t come up all that much, but I also almost never leave the continent.
esims solve the problem of instantly getting the sim card from the vendor to your hands with 0 shipping costs. This is especially apparent when traveling – the international esim market for travel is only really possible with esim technology.
It also enables some cool things like the ability to instantly switch carriers – I use the usmobile mvno as my primary carrier, and they offer the ability to near-instantly switch between t-mobile, verizon, and at&t; a feat that would be quite difficult with physical sims.
It’s also just more convenient to swap out sims – instead of fiddling with a sim remover and keeping track of tiny sim cards, I just instantly switch between sims in the settings screen (at least on ios). This is helpful because triple-sim devices are not common yet :)
They eliminate some friction when switching and eWaste. SIMs are fairly cheap to make, but they are ICs and have a non-trivial cost to produce and distribute.
SIM cards (which were credit-card sized) originally existed for car phones in rental cars. If you bought a SIM, you put it in your car phone and it became your phone. It’s at least 30 years since anyone cared about doing that. SIMs continued to exist because they’d been fairly good for ensuring handsets were portable across service providers. Once handsets supported SIM locking, that benefit went away and was replaced with regulations that required phone companies to unlock handsets.
The only reason to have a SIM now is to store a fairly short key. The key is small enough that it can fit on a QR code, so having a single-use IC to hold that key is incredibly wasteful. It also encourages lock in. To switch to a different phone provider (and, often, to switch between prepay and contract) you have to buy a new physical token in a shop or have it posted to you and then do a fiddly operation to remove the old one and insert the new one. For cheap pre-pay plans, providers often don’t assume that they can recover cost of the SIM from their profits and so charge for the SIM, which further adds friction to switching and encourages lock in.
With an eSIM or iSIM, the secure storage for the key is programmable and can hold multiple SIMs. If you want to switch plans, you just scan a QR code and now you have a new SIM loaded and your phone can dynamically switch between the one in use. If you’re travelling somewhere where roaming is expensive, you can get a local SIM before you arrive and start using it immediately. Some phones support more than one at a time, so you can use your normal SIM for incoming calls / SMS but the local one for data and outgoing calls. You can do that with a dual-SIM phone, but now you need to get the local SIM (airports have vending machines with a big markup, otherwise you need to find a shop once you’ve reached a town or city), not lose your old one when you install it, and so on.
Oh, and if you lose your phone or have it stolen, your provider can give you a new eSIM instantly, without having to get a physical one posted to you. This can be really important if you’re travelling and your credit card company uses SMS to notify you of potential fraud and declines transactions if you don’t reply: if your phone is stolen, you may be unable to use your credit card. Buy a new phone and get a new eSIM QR code and you’re back.
The only reason to have a SIM now is to store a fairly short key. The key is small enough that it can fit on a QR code, so having a single-use IC to hold that key is incredibly wasteful. It also encourages lock in. To switch to a different phone provider (and, often, to switch between prepay and contract) you have to buy a new physical token in a shop or have it posted to you and then do a fiddly operation to remove the old one and insert the new one. For cheap pre-pay plans, providers often don’t assume that they can recover cost of the SIM from their profits and so charge for the SIM, which further adds friction to switching and encourages lock in.
I don’t have access to the schematics and details for anything recent so taken with a few grains of salt, but when fingerprint authentication and contactless payment started coming in the android space, the fingerprint reader, HSM store, NFC and programs on the JVM inside the SIM were all coordinating within TrustZone to sign and authenticate the transaction.
In the 90s and early 00s, there was a window where saving contacts to my SIM card was the easiest way to move them between phones. Given that my phones at that time did not (readily) connect to my PCs, and they only had the standard phone keypad, I appreciated that convenience a couple of times.
“somewhat” cursed: Disable password authentication. If you have password reasons, leverage that x25519 private keys can be (almost) any 32 byte random string and kPub generation cheap/fast. Sha256(password) -> your private key.
I’m genuinely curious: how is a private key generated from a password more secure than the password? Is it just because no one is brute-forcing ssh private keys in that way yet or is there some cryptographic reason?
It’s not more secure in a traditional sense, the point is the password convenience without having the server expose password authentication as a supported method so any password guessers will leave it alone. In the nontraditional sense there are services that enforce public key authentication even though you don’t want to bother to supply it with one for other reasons.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I believe this is all of them, no? GitHub, GitLab, Codeberg, and Gitea all seem to have public keys available at /username.keys.
You’ve just given anyone who is aware that people might be doing this free reign to attempt to brute force your password hashes haven’t you?
Worse, I don’t think you can salt this scheme? So they can probably build a rainbow table esque structure and brute force everyone using this scheme simultaneously.
This is only true if you use the same SSH keypair for those services as you use for regular logins. My personal view is that you definitely shouldn’t, because attackers can use those public keys to probe for various things against your hosts (eg, a SSH client can find out if a SSH server accepts a given public key for authentication for a particular account).
(I’m the author of the linked-to article, but the information disclosures of known public keys is a completely different topic. In some circles it’s well known; see eg https://words.filippo.io/dispatches/whoami-updated/ )
I should have expanded the explanation of my view: for Github push keys and other things like it, I don’t think you need to go to the ‘generate private key through hashing’ approach. In most environments you’re extremely unlikely to wind up needing to push from a random machine that has not already been set up for this, so you don’t need to be able to produce a keypair from a seed phrase you can memorize. The only keypairs you need to be able to produce that way are keypairs you’d use from not set up machines.
(You can imagine scenarios where you might need to push something to make an urgent production deployment from an extremely minimally set up machine, but it’s a question of trade offs and how likely things are. Even then you might be better off with some kind of ‘break glass’ SSH keypair that is pre-deployed in encrypted form and maybe has to be enabled on the forge side before you can use it.)
If I was ever looking for an ‘out’ I think this was a fair trigger. A link to … a todo on a git forge with nothing of substance that hasn’t already been said a decade+ ago? Did I misspell phoronix.com? …
I’m not sure if it belongs here, but it’s not a TODO, it’s a WIP merge request currently with almost 2000 lines of new code that you can compile and try.
I’m surprised it works on Windows, because the kernel docs suggest it shouldn’t. DRM’d media is sent to the GPU as an encrypted stream, with the key securely exchanged between the GPU and the server. It’s decrypted as a special kind of texture and you can’t (at least in theory) copy that back, you can just composite it into frames that are sent to the display (the connection between the display and GPU is also end-to-end encrypted, though I believe this is completely broken).
My understanding of Widevine was that it required this trusted path to play HD content and would downgrade to SD if it didn’t exist.
If you have a path that goes from GPU texture back to the CPU, then you can feed this straight back into something that recompresses the video and save it. And I don’t know why you’d think this wouldn’t give you sound: secure path for the sound usually goes the same way, but most things also support sound via other paths because headphones typically don’t support the secure path. It’s trivial to write an Audio Unit for macOS that presents as an output device and writes audio to a file (several exist, I think there’s even an Apple-provided sample that does). That just leaves you having to synchronise the audio and video streams.
I’m pretty sure that what Gruber is describing is basically just “hardware acceleration is not being enabled on many Windows systems”, but because he has his own little narrative in his head he goes on about how somehow the Windows graphics stack must be less integrated. Windows is the primary platform for so much of this stuff!
I would discount this entire article’s technical contents and instead find some other source for finding out why this is the case.
Well it depends on the type of acceleration we’re speaking of. But I’ve tried forcing hardware acceleration on video decode and honestly you’d be surprised how much it failed and I did this on rather new hardware. It was actually shockingly unreliable. I’m fairly certain it’s significantly worse if you extend your view to older hardware and other vendors.
I’m also fairly sure, judging by people’s complaints, that throwing variable refresh rate, higher bit depths and hardware-accelerated scheduling in the mix has not resulted in neither flagship reliability or performance.
It can be the primary platform but this doesn’t mean it’s good or always does what it should or promises it’ll do.
Wait wait wait is this,,, checks URL, oh, lmao. Yeah Gruber is useless there’s literally no point in ever reading a single word he says.
I think it means: enabling the feature to screenshot DRM protected media would not by itself enable piracy, since people would not use screenshots to pirate media frame at a time.
What you are saying reads like “one technical implementation of allowing screenshots would enable piracy.” I trust that you’re probably right, but that doesn’t contradict the point that people would not use that UI affordance itself for piracy.
No one would use screenshots for piracy because all the DRM is already cracked. Every 4k Netflix, Disney, etc, show is already on piracy websites, and they’re not even re-encoded from the video output or anything, it’s straight up the original h264 or h265 video stream. Same with BluRays.
Yup, if you go through GitHub there are several reverse-engineered implementations of widevine, which just allow you to decrypt the video stream itself with no need to reencode. That then moves the hard part to getting the key - fairly easy to get the lower security ones since you can just root an Android device (and possibly even get it from Google’s official emulator? At least it supports playing widevine video!), the higher security ones are hardcoded into secure enclaves on the GPU/CPU/Video decoder though, but clearly people have found ways to extract them - those no-name TV streaming boxes don’t exactly have a good track record of security, so if I were to guess that’s where they’re getting the keys.
Still, no point blocking screenshots - pirates are already able to decrypt the video file itself which is way better than reencoding.
Those no-name TV streaming boxes usually use the vendor’s recommended way to do it, which is mostly secure, but it’s not super-unusual for provisioning data to be never deleted off the filesystem, even on big brand devices.
The bigger issue with the DRM ecosystem is that all it takes is for one secure enclave implementation to be cracked, and they have a near infinite series of keys to use. Do it on a popular device, and Google can’t revoke the entire series either.
Personally, I’m willing to bet the currently used L1 keys have come off Tegra based devices, since they have a compromised boot chain through the RCM exploit, as made famous by the Nintendo Switch.
Pre-DoD Speed Ripper early DVD rips jacked into a less-than-protected PowerDVD player and did just screenshot every frame.
This is interesting. Bubble implies a pop, yes?
I mean, things were overhyped and ridiculous, but can anyone say that the internet isn’t at the core of the economy?
Still one of the most widely used languages, powering many multi-billion dollar companies.
What we now just think of as “the web”.
Again, powering multi-billion dollar workloads, a major economic factor for top tech companies, massive innovation centers for new database technologies, etc.
Still massively important, much to everyone’s regret.
This one is interesting. I don’t have anything myself but most people I know have a Smart TV (I don’t get it tbh, an HDMI cord and a laptop seems infinitely better)
Still a thing, right? I mean, more than ever, probably.
Weirdly still a thing, but I see this as finally being relegated to what it was primarily good for (with regards to crypto) - crime.
Do we expect this “bubble” to “pop” like the others? If so, I expect AI to be a massive part of the industry in 20 years. No question, things ebb and flow, and some of that ebbing and flowing is extremely dramatic (dot com), but in all of these cases the technology has survived and in almost every case thrived.
All of these things produced real things that are still useful, more or less, but also were massively and absolutely overhyped. I’m looking at the level of the hype more than the level of the technology. Most of these things involved huge amounts of money being dumped into very dubious ventures, most of which has not been worth it, and several of them absolutely involved a nearly-audible pop that destroyed companies.
Yeah, I was just reflecting on the terminology. I’d never really seen someone list out so many examples before and I was struck by how successful and pervasive these technologies are. It makes me think that bubble is not the right word other than perhaps in the case of dot com where there was a very dramatic, bursty implosion.
The typical S-shaped logistic curves of exponential processes seeking (and always eventually finding!) new limits. The hype is just the noise of accelerating money. If you were riding one of these up and then it sort of leveled off unexpectedly, you might experience that as a “pop”.
See https://en.wikipedia.org/wiki/Gartner_hype_cycle
To me the distinguishing feature is the inflated expectations (such as NVidia’s stock price tripling within a year, despite them not really changing much as a company), followed by backlash and disillusionment (often social/cultural, such as few people wanting to associate with cryptobro’s, outside of their niche community). This is accompanied by vast amounts of investment money flooding into, and then out of, the portion of the industry in question both of which self-reinforce the swing-y tendency.
“Dumb TVs” are nigh-impossible to find, and significantly more expensive. Even if you don’t use the “Smart” features, they’ll be present (and spying).
Not for everyone and also not cheap, but many projectors come with something like android on a compute stick that is just plugged into the hdmi port, so unplug and it’s dumb.
Yeah, I’ve been eying a projector myself for a while now, but my wife is concerned about we’d be able to make the space dark enough for the image to be visible.
That’s assuming you make the mistake of connecting it to your network.
At least for now… once mobile data becomes cheap enough to get paid for using stolen personal data we are SO fucked.
Why have a TV though? Sports, maybe?
Multiplayer video games, and watching TV (not necessarily sports) with other people.
I use a monitor with my console for video games, same with watching TV with others. I think the only reason this wouldn’t work is if people just don’t use laptops or don’t like having to plug in? Or something idk
It’s the UX. Being able to watch a video with just your very same TV remote or a mobile phone it’s much much better than plugging your laptop with an HDMI cord. The same reason why still dedicated video game consoles exist even if there are devices like smartphones or computers that are technically just better. Now almost all TVs sold are Smart TVs, but even before, many people (like me) liked to buy TV boxes and TV dongles.
And that’s taking into account that a person own a laptop, because the number of people that doesn’t use PCs outside of work is increasing.
I have a dumb TV with a smart dongle - a jailbroken FireStick running LineageOS TV. The UX is a lot better than I’d have connecting a laptop, generally. If I were sticking to my own media collection, the difference might not be that big, but e.g. Netflix limits the resolution available to browsers, especially on Linux, compared to the app.
Citation needed? So far I only know about then either “stealing” existing tech as a service. Usually in an inferior way to self housing, usually lagging behind.
The other thing is putting whatever in-house DB they had and making it available. That was a one time thing though and since they largely predate the cloud I think it doesn’t make sense to call it innovation.
Yet another thing is a classic strategy of the big companies which is taking small companies or university projects and turn them into products
So I’d argue that innovations do end up in the could (duh!) it’s rarely ever driving them.
Maybe the major thing being around things like virtualization and related things but even here I can’t think of a particular one. All that stuff seems to steem largely from Xen which again originated at University.
As for bubbles: one could also argue that dotcom also still exists?
But I agree that hypes is a better term.
I wonder how many of these are as large because of the (unjustified part of) hype they received. I mean promises that were never kept and expectations that were never met, but investments (learning Java, writing java, making things cloud ready, making things depend on cloud tech, building Blockchain Knowhow, investing into currencies, etc) are the reason why they are still so big.
See Java. You learn that language at almost every university. All the companies learned you can get cheap labor right from University. It’s not about the language but about the economy that built around it. The same it’s true for many other hypes.
I have a list of one-off experiments that I pick something from whenever it feels like productivity is dipping. This weekend feels like one of those. The pick this time is to make a renderer that can output some point cloud visualisations of process memory for a Looking Glass portrait display the predecessor to the less interesting ‘cloud AI connected bullshittery’ (https://www.youtube.com/watch?v=N24CqEvtXxk).
For the unfamiliar, those are basically just a hidpi screen with a lenticular lens sheet glued on top. They hide a json blob in the serial number exposed over EDID with the calibration data (sheet properties and alignment). Realtime rendering to it is ‘a bit’ weird in that you encode the view angles you want into a ‘quilt’ so it’s n*m renderpasses and then postprocess in a shader that samples the quilt with offsets from the calibration.
My brother in christ, please just let me switch on strings.
The word “just” is sitting on top of a lot of complexity in this statement.
Pardon my lack of zig experience but I wouldnt expect any reference type to work in switch in a systems language (similar to C)
I would count rust as a systems language and it works in magch there (not just for &str but any reference).
Any constant reference.
Do you want equivalent but differently encoded unicode strings to be considered equal?
No, just raw byte-for-byte comparison.
None of the workarounds in the article seem to address that either.
Do you think that not allowing users to switch on strings will avoid canonicalisation issues somehow?
Can users define their own string type and make switch work on it?
would it make sense to allow an optional comparator to the switch statement? e.g.
switch (mystr, &std.mem.equal) { ... }I had been imagining something like the ability to extend
switchto operate on any aggregate type that has a hash-function definition available, but having a comparator generalizes that, and even supports some interesting use-cases like canonicalization-in-comparison (as @andrewrk asked about) as well as even fancier things like edit distance matching!I find this idea quite compelling…
All the best,
-HG
Sounds like pattern matching
On a multi-user machine
/tmpis shared. For downloads I would prefer some world-unreadable directory. Though I don’t even know what are the recommendations for things like that nowadays. On Linux there’s/run/user/<uid>, maybe this one? Doesxdghave any dir for “user’s temp files”?XDG_CACHE_HOME would be the closest match.
I know you’re replying to the last question of the parent comment, but, for downloads, if using XDG user directories, might as well use
XDG_DOWNLOAD_DIR.On macOS
$TMPDIRis a per user location unlike/tmp– although I suspect fewer macs are shared than Linux systems.This makes me curious. Do you do a lot of work like described in the article on a multi-user machine?
All my home systems are configured with NixOS and have users for everyone that might need to use them, so kind of yes.
But my PoV here is about doing things right w.r.t. security “and just in case”. If you have a system daemon that runs as a separate user, and it’s compromised, you don’t want it to snoop/modify your downloads etc.
There’s
$XDG_RUNTIME_DIR, which is usually/run/user/<uid>https://specifications.freedesktop.org/basedir-spec/latest/#basics
I think that’s a bad place for downloads, since the spec says
Practically speaking, on many distros this directory is a tmpfs a few GBs or so in size, so you do actually run into the size limit, it’s not just a theoretical thing.
On my client machines, I create ~/tmp/ and use that as a location to download, unzip source bundles when I only need a single file, throw a clone of a project for a comparison. I have it set up as a ram disk with a 7 day retention so it works more or less like tmp. Additionally, I set the noexec bit on the directory which makes it a better place for stuff that I just want to dig through.
A huge amount of the stuff that I bring onto my box comes through the ~/tmp/ folder first, and after looking at it and deciding I need it for longer I will move things to a disk some where.
I realized at some point that there’s a lot of stuff I know I’d toss in 2 hours but I also didn’t want to put it in actual tmp because it’s shared and I don’t want to mess with permissions or deleting it later.
In the ‘low mental bandwidth’ category I’m continuing work on frontending the Himalaya e-mail tool, current state: https://0x0.st/8ZoL.gif and hope to be daily driving after a few days more. It’s the first step towards something hopefully more interesting, and that’s part of merging a combined timeline across multiple internet socials (IRC, mastodon, lobste.rs, fossil, discord, RSS …) into non-intrusive monitoring.
For more focused work I am reworking IPC synch to (finally) drop the use of named primitives. The long holdout (outside of my general laziness on the topic) has been POSIX semaphores instead of FUTEXes thanks to the general footgun armory of all things system OSX. I have been hesitant to use the __ulock_wait/__ulock_wake facility for long enough that it is about time to take the plunge.
I didn’t use hotline, but lots of schools and local government used FirstClass which had similar vibes, dialup included. https://en.wikipedia.org/wiki/FirstClass#/media/File:FirstClass_GLFN_desktop.png
An anecdote from those times was that students had to copy the client from a public SMB share and run locally, while the teachers and admins ran it from the share directly with write permissions. The effect was that any messages or attachments they opened would write into temp files on that share yet those got world read permissions. So you could just scan/scrape that folder and get all the dish on who was flirting with whom or uncensored opinions on various students, password change responses, exams, …
Reminds me of Scour (out of UCLA) but with more general file utility
Finishing a note on untested.sonnet.io (no strong schedule this time, just writing for fun), then adding one small UI fix to the site. Done, in this exact order so I don’t get distracted by code.
Then messing about in the kitchen, I learned that Portuguese grelos is the same as the Neapolitan friarielli and there’s a guy on my street seeking literal heaps of it.
Playing Disco Elysium while the food is getting ready. Becoming a magnesium based life form.
Recently started Disco Elysium, fantastic game. Surprised by the lack of disco.
He is the guest in the podcast episode on “Cryptoanalyzing LLMs” which I consider worth a listening:
https://securitycryptographywhatever.com/2025/01/28/cryptanalyzing-llms-with-nicholas-carlini/
Yes, which is why I submitted it here. Good episode and also a good article imho :)
my favourite remains a shitpost of theirs, the “cryptography tier list” - a good use of AI voice cloning. I wish presidential discourse was at that level.
I hate saying this, but I’d love a TL;DR summary of these articles because they’re so lengthy. I’ve seen the others in this series posted here, and I still disagree with a lot of the premises. I don’t want clickable URLs in my shell buffer, I don’t think dragging a file onto the shell window should copy the file there, etc. There are just better ways of doing most of those things (IMO).
I don’t really get the focus on terminal emulation and TTYs. Nobody’s forcing you to use that stuff. There are a lot of ways to do “terminal stuff” outside of a terminal, with or without a CLI and TUI.
The obvious way would be to use Vim or Emacs and do the work there. Emacs has a TUI and can (or can be made to) do all of the things the author is asking for, and the underlying details of TTY protocols and terminal emulation is entirely hidden away.
Honestly, besides
emacs, do you have anything else on your mind?I’m an emacs user, so that’s the only one I have much direct experience with.
I’d imagine most editors can do it. I’ve seen a half-dozen “magit for $something” articles posted here, so probably those tools. VSCode has a shell/terminal thing.
Vscode’s terminal is a regular terminal emulator.
It doesn’t mean it’s correct, though. I like the the separation of concerns, and so an editor should be an editor. What people are doing in the e.g. Neovim community is insane to me (package managers, many Lua rewrites of tools you can use in the CLI, etc.).
I think the evolution needs to come for « CLI replacements », and so far besides Cat9 I really haven’t see anything promising (besides emacs, but it comes with its own issues).
We likely consume, and expect to consume, content on the Internet in very different ways. I personally don’t deal with short-form whatevers at all. If it’s not worth at least an hour with a printout in the arm chair I simply stay away.
You are forced to use terminal emulation and ttys every step of the way (assuming Linux/BSD). The only one that successfully paved another way is Android and it worked rather well for them, and even then it pops up here and there. I know, I worked on it. The first thing your desktop has to do in order to get something in the screen is a ridiculous dance to tell the TTY to please stop doing its thing, and even then it’s there in ways that completely undoes the security model they try to implement.
That something is ‘hidden away’ doesn’t mean that it doesn’t influence every layer above it. Emacs pays through the nose to get its TUI to work. There’s a ridiculous amount of code and workarounds to present its rather modest UI.
A tangent perhaps but as a former Emacs user and as former Vim user (neither are even close to doing the things I ask for) I find it fairly bizarre to use a text editor to do shell work and vice versa. Just as I avoid trying to use a tooth brush to clean my toilet or a toilet brush to clean my teeth. They can get the job done, sortof, but at a price and with some risks.
I’m going to disagree with your last paragraph and say it’s quite natural to use a text editor to do shell work.
Most “shell work” involves navigating around, searching through, and otherwise working with text, which is exactly the toolset built into text editors. cat, grep, sed, and awk are command line tools implementing text editor functionality in a clunky (clunky for interactive use) pipeline/streaming fashion. Likewise, ls, find, cd, and other filesystem navigation is also basic text editor functionality. They all spit out text, and what better way to view and work with it than a text editor? Maybe I’ve had too much Emacs koolaid, though.
For the rest of it, I don’t care much. On a day-to-day basis I never have problems related to TTYs, and with a few (easily fixed) exceptions in the past, it’s been that way for a long time.
With that logic you can use a hex editor for everything, it’s all “just” bytes no? representations matter, interventions matter. Trivialities aside, they operate on different mechanisms and assumptions. Text editing is buffers with some intended persistence and at rather small buffer sizes. You can expect to seek and build features assuming that works; the backing store changing is an exception, not the default and so on. The cli shell is stream processing. Its data sources are hierarchies of short lived processes that might expect something back from you to do their job and can spit out infinite amounts of data. The world can change beneath its feet between every command. To do that it gets to go through all the ugly legacy – signals, sessions, buffer bloat, line blocking and with that, serial communication.
This brings us back to never having problems; ‘works for me’-isms is just dismissed with ‘doesn’t work for me’-isms and doesn’t get you anywhere. It’s systems engineering, one doesn’t get to think like that. You don’t have to dig far to find people with serious problems with the TTY and a few vulnerabilities a year. First(?) Flatpack sandbox escape? the tty again.
Previous maintainer of the subsystem? Original gangster Alan Cox - https://lwn.net/Articles/343828 quit (incidentally triggered by a bug from Emacs losing data while trying to be a shell) Greg tried to step up, well - https://www.youtube.com/watch?v=g4sZUBS57OQ
Whenever they try to fix something to make embedded development bearable, you commit the Linus taboo of breaking userspace because your stack relies on hidden assumptions that all go back to the emulating the fantasy computer. To edit text. It’s holding everyone back.
Just the other week I had a coffee spill. It fried my mouse and worse, I had to make a new cup of coffee. I grabbed another rodent from the box of spare input devices. I plugged it in. Now my keyboard stopped working. I unplugged the mouse again. The kernel crashed and I lost a lot of work. Angry reboot. Plug back in. Now there’s half a second delay between keypresses every 10 seconds. 2 minutes later the machine froze up. It turned out the USB mouse exposed itself as serial rather than USB human-interface. This meant some /dev/input/eventN gets routed through the tty layer again. The display server ran the device node non-blocking, but that only goes skin deep. You still have ‘the global’ lock around in tty and a handful of mutexes on top of that. Race condition into memory corruption and panic. Then race condition into deadlock.
The point you missed is that a text editor has tools to work with text, while hex editors typically do not.
I find working inside of emacs generally easier than in the terminal because it has great tools for working with buffers of text, which is what most shell commands produce. If there were a hex editor that made it even easier, then I would indeed use it instead.
This is just moving from the unix-style cli to the old win32-style cli. Except Microsoft gave up and ended up embracing ANSI escapes and CSIs (some time after WSL landed) to support the ever-growing corpus of apps specifically targeting terminal emulators.
In the Win32 console model, apps emitted only text to stdout and used the console handle to do everything else from setting attributes to switching between the full screen buffers. And the open source world made fun of them for it.
I wouldn’t do that, it isn’t like the unix terminal design is exactly ideal either.
I don’t think the unix design is as offensive on the output side as the input though; if you’re just writing a program that wants to output some colorful text, it isn’t all bad. Just remember to check isatty and the command line flag that overrides that before adding the color sequences!
On the input side though, yikes, what a pile of disgusting hacks. You have to do a timeout check on the esc char! And woe be upon you if you need to query the terminal for something, since now that’s both input and output sequences! Worth noting a lot of the newer extended terminal emulators are introducing new features to change this stuff, but they’re tied by the need to be compatible with the large existing ecosystem of applications.
There’s also the separate question of api vs underling data too. In the underlying data, one stream does make synchronization and redirection and other things easier. But then in the api you might want it separate anyway, to make supporting
--color=autoand--color=offeasier. Stripping escape sequences from the output isn’t awful though, so you could just always write them in your code and have the library function just does that.Nevertheless, I’m not a fan of making fun of designs… there’s often fair reasons for differences that you can learn from.
If you generalise to that level, well, duh. The argument that in-band signalling UI is harmful in the same way smashing strings together versus prepared statements is not a difficult one.
Microsoft didn’t “give up” – it was a no brainer for them to throw a few idle headcounts on writing an emulator for their WSL trajectory and getting more developers into VScode. They’re otherwise busy adding AI keys and ads in the UI so perhaps their sensibilities is not something to use as ‘appeal to authority’ or inspiration.
I am intimately familiar with the inner workings of the different console layers in the MS ecosystem and their reasons for being – from the days of conio.h, int 21h, mode con codepage prepare and onwards. Their powershell model is much closer in comparison but the power they had there came specifically from them having something of a functional IPC system to work with. Linux/BSD isn’t so lucky.
Someone should write some Python bindings, an app, and report back to us on the experience!
So someone did many years ago, it was in a startup that collapsed just before the finish line (2020 march, thanks covid). https://github.com/letoram/tui-bindings/tree/master/attic/python
It was a junior being thrown into the deep end of the pool (at his request) and had zero experience with the API (unsurprising) but also with python bindings. It’s not been kept up to date as my personal policy with python is that if something is written in it I don’t use it unless at gunpoint.
We used it for a monitoring system for a turnkey distributed ‘from breakpoint in vscode / .qcow2 upload in webUI’ → ‘analysis / harness generation’ → distributed across cluster → crash collection into triaging → UI callback’.
What someone should do though, is to take the -server end of this, embed into whatever terminal emulator is in fashion and suddenly there’s a seamless transition path to bootstrapping in other desktop environments outside of Arcan.
This is IMO a great example of why process isolation is the only path to confidentiality on modern CPUs. Site- (or origin-) process isolation has been and continues to be the durable path forward here.
It’s even directly mentioned: “our proof-of-concept opens Proton Mail’s inbox in a new window, but uses the same process to render the inbox” (emphasis mine).
Also note: the CPU is not the problem here IMO, it’s doing exactly its job.
Yes 100%, good quote from HN: https://news.ycombinator.com/item?id=42856501
Process isolation seems to be poorly understood
I have been getting push back on it in this thread today: https://lobste.rs/s/utwbwv/can_we_retain_benefits_transitive
And in the xz backdoor threads: https://lobste.rs/s/uihyvs/backdoor_upstream_xz_liblzma_leading_ssh#c_wgmyzf
Pointing this out again:
Ubuntu 24.04 (and Debian) removed libsystemd from SSH server dependencies - https://news.ycombinator.com/item?id=40018925
i.e. an actual fix for the xz backdoor was to move code into different processes; to prevent Jia Tan code from sharing an address space with sshd code
(This was about a backdoor, not information leakage, but I think it’s still relevant – process isolation mitigates many different attacks)
I think a main issue is that process separation can introduce some nasty programming problems, making life harder for the developer (not to mention portability problems)
But it makes life better for the user, which should be the goal
So hopefully we will continue to find ways to make it easier, to have the best of both worlds
I have been isolating parser -> render now since about 2008. After Spectre I went ‘oh this is not enough’ and started using 1 task, 1 device then re-image and reboot.
A previous side-hustle (2015 or so) was a plugin that did that to ‘open attachment’ at a law firm that had more phishing attempts than actual e-mails. A pool of runner VMs in their cluster, each attachment got a fresh image. The weak link in the chain was the dependency to printers still but that at least was monitored heavily anyway.
The intro seemed fine, but I stopped reading at
Fuck it. The image is unnecessary, ugly and my reading experience is made worse by its presence. I’ll follow the recommendations from the cryptography blog with the beautiful furry art instead.
That key image literally broke my pattern recognition (as many GenAI images do) do this point of my eyes unable to focus on the actual page and had to print to text and read it that way. Only other time I have had such a strong reaction was when playing around with an eyetracker and using the gazepoint to warp the mouse cursor. A really bad idea and I think I know how a cat feels around laser pointers now.
That aside, the recommendations are solid and the writeup good (as expected from trailofbits).
I personally just glazed over it to continue reading.
Didn’t notice it till I scrolled up again because I saw “figure 2” and thought..
“Hmm what was “figure 1” I don’t remember…oh! It wasn’t important.. Oh that’s AI generated? Okay”
IMO it’s a pointless addition, whether AI generated or not.
I don’t think we should blame (only) the authors, but also the ecosystem, because at the moment every “social site” practically requires an image to be shown as the “snapshot” of a shared link, thus the following HTML meta-tag which happens to use the same image as the first one in the article:
What happens if you don’t include such an image for each and every page of your site? Technically nothing, the social site just puts a blank image, a placeholder, or a scaled-up variant of the site’s favicon. However in practical terms, I bet the link is downgraded by “the algorithm”.
Thus, just ignore the blurb images, as I do almost every other site out there, as they are a requirement in today’s web-conomy.
I get you, but it could also be any image, any image at all. I often just use vacation photos. There is no requirement that it be a distracting bad AI image. The flippant use of an AI image is a strong quality indicator that implies the rest of the post will be garbage too. Better for the authors to have put in some effort.
Another piece of the web-conomy machine you are dismissing is “copyright” and “licensing”. Sure, for you blog, you can use your own vacation photos, because it’s not like you’re going to sue yourself for using them; but this is a company’s blog, written by a company employee (or contractor as it’s most often the case), thus he can’t use photos from his own vacation, because then he also has to transfer copyright, or at least license that image for the company to use, thus lawyers. Another option is to use some stock photo, but that means money, and it’s not like a bland stock-photo image is better than a LLM regurcitation. Thus, the cheapest solution is to make the LLM spit out some blurb image that has no copyright, thus no legal entanglements.
I can assure you that the post text is not garbage. It has a few good points that are expressed in plain words, instead of the crypto-highfalutin language other cryptographers use in their blog posts.
Unsplash is a site with CC0-licensed images.
No yeah, I agree, the article is fine, it’s just the use of AI slop ruined any first impression, and I can see why many people didn’t bother to read any further. It probably did seem cheap to the authors at the time, and that’s a shame.
This is a very poor excuse for bad behaviour:
og:imageif the article does not have one. It can be a properly sized logo, or a photo of the author. If Wordpress does not support this, I guess there’s a plugin to implement it.This is super interesting but I’m having a hard time understanding the full extent of the post. Is there a project implementation somewhere?
I’ve been thinking strongly along the same lines for a while now. We need to break free of terminal emulation and in-band signaling for text based interfaces. I’d be super curious to read more about efforts in this area.
It’s distilled experience from 10 years of slowly sifting through the ~130kloc of linux-tty; 200kloc of ncurses; 70kloc of GNU readline; 70kloc of tmux; 150kloc of openssh + a handful of emulators (excluding all other display server things) – then abstracting and implementing. All because a former colleague dared me to. Now I can claim my beer. That’s to say there is quite a few moments of ‘oh no’ that change your mental model of things that’s hard to convey without some serious code-fu.
There’s quite a few people with insight and experience by now on the Discord (dang kids rejecting IRC).
As for other examples on it being used in lieu of ncurses: https://github.com/cipharius/kakoune-arcan/tree/main https://github.com/letoram/nvim-arcan
I recall someone reproducing ACME in Rust as well, but I can’t seem to find the link, it was very “on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.”
IRC could have progressed, but it stayed a fossil, ignoring all technological advancement in the past 30 years.
Yeah. Telling users “if you want message history or to be able to receive DMs while not logged in, just run your own bouncer on a server somewhere” is ridiculous.
I’m shocked that Arcan uses Discord. I thought free software projects were the last bastion against it and of all the projects I would never have expected Arcan to capitulate…
Understand the enemy and get behind enemy lines, talk to the locals and gather intelligence. I don’t plan on staying there for longer than necessary but the alternative is early in proof of concept. In the current slopjob of an Internet I see Discord as one of the more dangerous of accidents (accident in the sense of its original scope of coordination/paratext around gaming).
The quality of information and community if you know which ‘fake BBS’ (sell the appearance of distributed, ‘click to spawn community infrastructure’) to look for is something I’ve not seen elsewhere, not on IRC, not on Usenet, not in mailing lists and so on. Their owners are getting a very good training corpus when other wells are poisoned.
A case worth looking at is the Windows console api…. which moved in the opposite direction. They had something free of that stuff and went toward terminal emulation, mostly in the name of compatibility with the existing ecosystem of terminal stuff. That’s a strong pull, and hard to imagine adoption of something without two way compatibility there, which does limit your design space somewhat.
But the Windows api - much maligned by some (and tbh its default user interface is kinda meh), but i like it - is more to the PC hardware text mode what xterm is to the vt100. You see the design influence and there’s some emulated compatibility, but also new concepts layered on top that weren’t in the original hardware (of if it was, i didn’t know how to use it lol).
In the hardware text mode, the screen buffer was composed of 2 bytes per character cell. One was the character code, which mapped to the font, the other an “attributes” byte, which was packed bits of foreground color, background color, and blink. Being an in-memory buffer, of course, you can poke those bytes in any random order or do block copies or whatever, but the print apis would generally just write the characters with a default attribute byte. Input was provided entirely separately since they’re separate hardware.
In the win32 console, they expanded on this: the character and attribute thing there is 16 bit, but same idea, they associated a current attribute with the screen buffer then provided functions to write text separate from change attribute, generally abstracting it into functions rather than an exposed memory block (though you can ask it to copy the memory block in and out for you - see ReadConsoleOutput), but you can see the heritage pretty clearly. So to write out some colored text, you first
SetConsoleTextAttribute(handle, red);thenWriteConsole(handle, "text");. This drives people coming from linux nuts because they so badly want to justprintf("\033[44mtext\033[49m")… but there’s a lot to like about separating those things.The win32 console also lets you allocate entirely separate screen buffers, write to them in the background, then swap it out to be displayed. Doing this with a unix terminal is…. non-trivial, and this is one of the functions they just plain left behind in their transition to unix style. But it is very useful, letting a program work in relative isolation. (In unix terminals, they usually set the magic code for “alternate screen”, vim does this for example, but there’s just one alternate screen, you can’t have multiple that you swap around.)
Input also works differently than unix: there’s two modes.
ReadConsoleInputgives you an array of structured input events. Key down, key up, mouse moved, window resized, etc., all different tagged structs. MUCH nicer than reading random sequences on a timer and decoding them according to random environment variables! You can loop on these like any other event loop and find joy. Or you canReadConsole, which just picks the characters out of it.ReadConsole is interesting too because when you enable line buffering, the system provides a line editor for the user - arrow keys work, backspace works, there’s programmable history (Get/SetConsoleHistoryInfo), there’s even a built-in tab completion hook! (I could never find decent documentation of this so i wrote something myself on stack overflow then expanded on my blog https://dpldocs.info/this-week-in-d/Blog.Posted_2019_11_25.html#tab-completion i got an email once from a guy who said he worked at Microsoft who said that was the best doc he could find on it too lol). In theory, you can do some of this on a linux terminal too - cooked mode read line is buffered but it is more under the kernel’s control than either the terminal or the application, hence why most everyone uses gnu getline (or one of its many similar alternatives) instead when they want a richer or customizable user experience, but the Windows API is reasonably rich and customizable out of the box.
All this said… look up the docs to those functions, and every one of them has a deprecation warning, saying use the virtual terminal sequences instead. Sigh, feel like a step backward. But I understand why they did it given the ecosystem pressure.
Still, the Windows API is worth studying as another way to break free of these terminals, and from what I’ve seen of Arcan, in this post and others, looks like they’ve studied it too.
Back in the mid-90s, I wrote my own “terminal API” for the PC text screen (my own TUI as it were). You could define a region of the screen (default region—the entire screen) and do things like
ScrollUp(region,n)(to scroll the region up n lines) orScrollLeft(region,y)(to scroll the region left y lines) or evenSetFGColor(region,c). At the time, I didn’t support separate screen buffers, but I think it could have been added with no issue.And I liked this API, but how well would it lend itself to a “remote screen”, say, across a serial or network port? Some form of protocol would have to be defined to separate the commands from the data. And as for supporting existing physical terminals, there would have to have been a translation layer because I don’t know of any physical terminal supporting left/right scrolling (although I would love to learn othewise).
When Unix was being developed, there were dozens of different physical terminals available, all working over serial ports (the USB of the day). But the command sets and features vary wildly. The ADM-3A was a very popular (read: cheap) terminal, but it’s capabilities were extremely limited (you could move the cursor, clear the screen, and scroll the entire screen up—that’s about it). But the VT-100 was way more capable, and later models even more so. Unix solved this with a library, using an environment variable to select the terminal type and managing the differences so you didn’t have to.
The only reason a timer is required is because of the
ESCkey—you need to determine if anESCcharacter is a key press, or part of a control sequence. It would have been nice had terminals sent a control sequence basically meaning “ESCwas pressed” then all the timing would have been avoided, but alas, we have to use the terminals we got, not the terminals we want.Indeed… and they actually still kinda do, but you can’t use TERM since everybody just claims to be xterm to avoid being treated like a dumb featureless thing by ncurses. Reminds me of User-Agent: Mozilla 5.0 (compatible; MSIE….) in the browser world. Environment variables are kinda meh for this anyway (whether TERM or the TERMINFO ones) since you might move between terminals and need to adjust too; screen, tmux, etc. of course can offer a layer of stability but it’d be nice sometimes to just send down a “relocated” event with new information. (I guess in theory you could hot-plug hardware terminals too, but in practice that’d send a SIGHUP surely anyway.)
Aye. Even now, 50 years later, we’re stuck with the terminals we got. There’s some newer ones that will send the CSI character instead exclusively, but you need to detect and swap and probably still check the esc char for compatibility anyway so, partial success at best.
I hope some day something like Arcan can manage to finally deliver a terminal we want, but it is definitely an uphill battle.
You have other timing channels on the input end - in the sense of the instruction decoder needing to do something when there is nothing in the immediate read buffer after an ESC, OSC, DSC and so on – even though that can happen naturally from a network stall / dropped packet retransmission / … It ends up in a heuristic in the emulator and tty implementation: do you consider blocking until you get more even though that would interfere with the software you are hosting?. You don’t notice it as clearly as with the ESC keypress (we are quite sensitive to jitter and judder). Networks are much faster these days and ncurses et al. tried to smooth it over by using a bandwidth estimator against baudrate so they don’t emit outputs until it’s unlikely that a sequence would get fragmented. I’ve run into plenty of cases where inputs and commands gets misinterpreted one way or another on slow serial links trying to provide a TUI (network routers, debug interfaces).
On the output end you have other timing breaks, an easy case is looking at how an emulator is handling “find /”. You can’t block until it’s completed or people complain you’re unresponsive. If you align to some artificial vsync from outer display system others will complain that you are ‘slow’ when they benchmark emulator a vs emulator b by timing how long a command will take (yet its actually the heuristic being measured). At the same time you don’t know what is actually running because a shell is in the way, so another set of heuristics depending on if you are in altscreen versus line mode and so on. This breaks when you nest terminal emulators (as the inner one becomes altscreen to the outer one even though the actual command is in line mode). Compare running find ‘raw’ and within tmux.
On the sideband end you have others in the signal propagation mess, obvious one is SIGWINCH on drag resize; be too quick to process and propagate resize events and contents will tear and corrupt endlessly. Be too slow and it will feel laggy. This is one you try to mask by padding with background colour and retaining ‘old’ contents hoping the user won’t notice as well as have rate limit or bin on something like the X11 XA_WM_SIZE_HINT width_inc and height_inc matched to cell sizes…
Oh dear, limited colours, no emoji … does it even underline/bold/italic?
No, but you can represent the associated emphasis with color (and on Windows, you can query the background color to adapt accordingly!). This is done often on many linux terminal emulators too. For example, mine does underline, but ignores italic and treats bold as just a color modifier. xterm does bold, but treats underline and italic i think as a color change (that might be configurable im not sure).
Until Vista, you could hit alt+enter and bring a console full screen, using the actual hardware text mode too.
Of course, extending it to support these things would not be difficult (in theory), there are some bits available lol. (Tho if I was redoing it, I’d probably not keep the char/attributes packed like that, and instead put the attributes as a separate set of overlay ranges, more like a rich text component’s innards.)
Depending on when the API was defined, Unicode might not have been a thing at all (Windows 1.0 was released in 1985 after all). And it wasn’t until the mid-to-late 90s that you got sufficiently high resolution with more than 256 colors on PCs (yes, VGA, available in 1987, supported a 256-color mode, but that was only at 320x200). It was also probably developed to handle both text mode and graphics, thus some of the limitations due to text mode.
The reason I commented on the limitations is that it is so obviously constricted by the time it was designed, without leaving any space to grow.
It’s sad that the most prominent alternative to the terminal interface had such lack of foresight, and the supposedly worse technology has been able to incrementally adapt from 1 bit to 4 bit to 8 bit and then 24 bit colour, from ASCII to Latin-1 to unicode with bidi and emoji.
There’s something weird going on here that I’m struggling to articulate. I think there’s an uncanny valley between the strict limitations of a terminal and the wide open possibilities of a GUI (or the web), where there isn’t a happy medium that’s as easy to program as a terminal but more graphical or more reactive or something.
I admire people who are trying to find or make that happy medium, but I fear the opposing pulls to the terminal or the web are so strong that building a middle ground is much harder than we might hope.
There’s another weird thing going on here too … stuck with sub-optimal solution because of backwards compatible (probably for things you don’t really want to concern yourself with) and being forced on the upgrade treadmill (because you want the feature—it’s new! It’s fresh! It’s the new shiny!).
Another issue is that it’s easier to combine command line programs (shell scripting) than it is to combine GUI programs. While there have been attempts to script GUIs (Amiga with REXX, Apple with AppleScript) none have really caught on. I have a feeling it’s because most people aren’t computer literate (computerate?), nor are they taught (in my opinion) true computer literacy (which is “computers can do repetitive tasks”) and GUIs are not really geared for it either.
I think we share a similar feeling, though I am far from finished describing or demonstrating it. My first attempt at jotting it down: https://www.divergent-desktop.org/blog/2020/08/10/principles-overview/#p3 with https://arcan-fe.com/2021/04/12/introducing-pipeworld/ an experiment to understand how interchange could work.
The API behind the appls of Arcan is a long running attempt of finding such a thing, and I’m fairly content with the capabilities by now, but it took a lot of dogfooding. In that sense getting rid of curses and terminal emulation was a side-quest.
I think this API was Windows NT, but I’m not entirely sure. It does use Unicode nowadays at least via the 16 bit characters.
There are links to the code at the top of the page; see “Code (Fossil)” for the main repository, and “Code (Github)” for the github mirror.
Still waiting for an affordable FIB-SEM to magically appear in my local bargain bin.
What surprises me more is the very existence of Electron. There ought to be a package format for the ‘app’ part and chrome itself capable of launching it. Having a fork that drags behind for such a massive and evolving codebase just hurts packaging and increases the value of n-day exploits.
Those were “Chrome Apps” which almost nobody used. I believe that was in part because Google kinda led people to believe they had to distribute those apps through the Chrome web store. That wasn’t really technically true but the developer documentation all assumed it.
One of Electron’s big draws is making it cheap to write a single app that supports the two most popular desktop operating systems, Windows and macOS. So I’m not at all surprised that no such package format exists, given that the format’s author would have to convince Microsoft and Apple to support the format.
macOS’s closest thing to a package manager is the Mac App Store. It has a lot of limitations, including not supporting third-party registries. So Apple is far from being able to support a specific package format.
Microsoft supporting a new format would be more plausible. Windows finally added the Windows Package Manager to the OS in 2020, seven years after the release of Electron.
Microsoft basically does support the web app stuff. Their WebView2 is an auto-updating embeddable chromium/edge for applications to use and you can declare your intention to use it with a little MS-provided dll or in the packager of the IDE (I know that’s vague, I’m not an IDE users, but I’ve seen the checkboxes in there before).
I know this isn’t exactly what you have in mind, since you still package an exe and dll, but it really isn’t awful.
I don’t think this is true. It would be better that way, sure, but you can do it in userland. Instead of embedding Chromium the Electron app would embed a thin, infrequently updated runtime management layer that would manage the Chromium runtime in some common directory that every app used.
Lots of flaws with this approach obviously, many that would be solved by the system managing it, but I think most if not all of them are solvable decently satisfactorily.
Having Microsoft support it would be in their interests as they can then route it from App Store through Edge. That is enough surface area for it to be relevant.
Can someone ELI5 why I should care about esims? What actual problems of actual users do they address?
My (perhaps uninformed) take was simply that physical sims were a victim of Apple’s quest to have the iphone look as much as possible like a solid seamless block of glass and aluminum with nary a button nor port to be seen. Would I actually benefit in any way from using an esim in a device with a physical sim slot?
When traveling, especially to remote locations, obtaining an eSIM before the trip can be more convenient than trying to obtain a card locally, again especially if you want a specific plan (vs whatever a local shop might stock). Some plans are only available as eSIMs, so if you want one of these being able to do it in any device is useful. Very few phones can handle more than 2 physical SIMs and swapping them is kind of annoying, many phones can store more eSIMs - how much you need that again depends on your travel and usage patterns, for me it would’ve been quite nice before free EU roaming became standard, nowadays it doesn’t come up all that much, but I also almost never leave the continent.
esims solve the problem of instantly getting the sim card from the vendor to your hands with 0 shipping costs. This is especially apparent when traveling – the international esim market for travel is only really possible with esim technology.
It also enables some cool things like the ability to instantly switch carriers – I use the usmobile mvno as my primary carrier, and they offer the ability to near-instantly switch between t-mobile, verizon, and at&t; a feat that would be quite difficult with physical sims.
It’s also just more convenient to swap out sims – instead of fiddling with a sim remover and keeping track of tiny sim cards, I just instantly switch between sims in the settings screen (at least on ios). This is helpful because triple-sim devices are not common yet :)
They eliminate some friction when switching and eWaste. SIMs are fairly cheap to make, but they are ICs and have a non-trivial cost to produce and distribute.
SIM cards (which were credit-card sized) originally existed for car phones in rental cars. If you bought a SIM, you put it in your car phone and it became your phone. It’s at least 30 years since anyone cared about doing that. SIMs continued to exist because they’d been fairly good for ensuring handsets were portable across service providers. Once handsets supported SIM locking, that benefit went away and was replaced with regulations that required phone companies to unlock handsets.
The only reason to have a SIM now is to store a fairly short key. The key is small enough that it can fit on a QR code, so having a single-use IC to hold that key is incredibly wasteful. It also encourages lock in. To switch to a different phone provider (and, often, to switch between prepay and contract) you have to buy a new physical token in a shop or have it posted to you and then do a fiddly operation to remove the old one and insert the new one. For cheap pre-pay plans, providers often don’t assume that they can recover cost of the SIM from their profits and so charge for the SIM, which further adds friction to switching and encourages lock in.
With an eSIM or iSIM, the secure storage for the key is programmable and can hold multiple SIMs. If you want to switch plans, you just scan a QR code and now you have a new SIM loaded and your phone can dynamically switch between the one in use. If you’re travelling somewhere where roaming is expensive, you can get a local SIM before you arrive and start using it immediately. Some phones support more than one at a time, so you can use your normal SIM for incoming calls / SMS but the local one for data and outgoing calls. You can do that with a dual-SIM phone, but now you need to get the local SIM (airports have vending machines with a big markup, otherwise you need to find a shop once you’ve reached a town or city), not lose your old one when you install it, and so on.
Oh, and if you lose your phone or have it stolen, your provider can give you a new eSIM instantly, without having to get a physical one posted to you. This can be really important if you’re travelling and your credit card company uses SMS to notify you of potential fraud and declines transactions if you don’t reply: if your phone is stolen, you may be unable to use your credit card. Buy a new phone and get a new eSIM QR code and you’re back.
I don’t have access to the schematics and details for anything recent so taken with a few grains of salt, but when fingerprint authentication and contactless payment started coming in the android space, the fingerprint reader, HSM store, NFC and programs on the JVM inside the SIM were all coordinating within TrustZone to sign and authenticate the transaction.
In the 90s and early 00s, there was a window where saving contacts to my SIM card was the easiest way to move them between phones. Given that my phones at that time did not (readily) connect to my PCs, and they only had the standard phone keypad, I appreciated that convenience a couple of times.
“somewhat” cursed: Disable password authentication. If you have password reasons, leverage that x25519 private keys can be (almost) any 32 byte random string and kPub generation cheap/fast. Sha256(password) -> your private key.
I’m genuinely curious: how is a private key generated from a password more secure than the password? Is it just because no one is brute-forcing ssh private keys in that way yet or is there some cryptographic reason?
I make no comments on the overall soundness of the idea, but I think it addresses the password snooping issue I mentioned in https://lobste.rs/s/gvo8fy/thoughts_on_having_ssh_allow_password#c_nkj4ho .
Pubkey auth doesn’t exist the key to the server.
It’s not more secure in a traditional sense, the point is the password convenience without having the server expose password authentication as a supported method so any password guessers will leave it alone. In the nontraditional sense there are services that enforce public key authentication even though you don’t want to bother to supply it with one for other reasons.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I believe this is all of them, no? GitHub, GitLab, Codeberg, and Gitea all seem to have public keys available at
/username.keys.If public keys can be scraped…
You’ve just given anyone who is aware that people might be doing this free reign to attempt to brute force your password hashes haven’t you?
Worse, I don’t think you can salt this scheme? So they can probably build a rainbow table esque structure and brute force everyone using this scheme simultaneously.
This is only true if you use the same SSH keypair for those services as you use for regular logins. My personal view is that you definitely shouldn’t, because attackers can use those public keys to probe for various things against your hosts (eg, a SSH client can find out if a SSH server accepts a given public key for authentication for a particular account).
(I’m the author of the linked-to article, but the information disclosures of known public keys is a completely different topic. In some circles it’s well known; see eg https://words.filippo.io/dispatches/whoami-updated/ )
Sure, you can reduce the damage by only using this keypair for certain things.
If someone can reverse my GitHub push private key though… that would be pretty bad all by itself.
I should have expanded the explanation of my view: for Github push keys and other things like it, I don’t think you need to go to the ‘generate private key through hashing’ approach. In most environments you’re extremely unlikely to wind up needing to push from a random machine that has not already been set up for this, so you don’t need to be able to produce a keypair from a seed phrase you can memorize. The only keypairs you need to be able to produce that way are keypairs you’d use from not set up machines.
(You can imagine scenarios where you might need to push something to make an urgent production deployment from an extremely minimally set up machine, but it’s a question of trade offs and how likely things are. Even then you might be better off with some kind of ‘break glass’ SSH keypair that is pre-deployed in encrypted form and maybe has to be enabled on the forge side before you can use it.)
There’s more than one hash function to chose from. You can run n number of rounds of it.