The writing is all me. While I was writing it I had the idea for that Colophon feature, so I ended up spending half an hour with an LLM to build that tool and create that section of the post.
On a previous post from @simonw, you made a similar snappy comment which later you deleted after others politely replied you. Now you do it again, strangely targetting personally the author. I don’t think this style of communication fits this community.
This is the handout for a workshop I gave this morning at the NICAR data journalism conference. The handout is designed to be useful completely independently of the in-person workshop itself.
I wrote a bit more about this workshop here, including notes on the various new pieces of software I developed in advance of teaching at NICAR.
I originally planned to include notes on running local models for structured data extraction - but I ran into a problem with length limits. For data extraction you need to be able to input quite a lot of text, and the current batch of local model solutions (at least that I’ve tried) tend to stop working well once you go above a few thousand input tokens. Meanwhile OpenAI handles 100,000 and Gemini handles 1-2 million.
In journalism you’re often working with public data, in which case I see no problem at all feeding it into models.
I had some interesting conversations at the conference about the much larger challenge of analyzing private leaks. Those are cases where dumping them into an online model isn’t acceptable - the best solution right now may be investing in a new $10,000 Mac Studio and running capable long context models on that.
Those are cases where dumping them into an online model isn’t acceptable - the best solution right now may be investing in a new $10,000 Mac Studio and running capable long context models on that.
What are the most capable long context models nowadays that you can run locally? And why do you specifically need a Mac Studio for that — I’m assuming for the ludicrous amounts of shared memory? How much exactly of it would one need?
That’s something I’ve been trying to figure out. The problem is that longer context requires a LOT of both processing and memory.
I ran a survey about this yesterday on Twitter and on Bluesky - in both cases I asked:
Anyone had much success running long context prompts through local LLMs? On my M2 64GB Mac I’m finding that longer prompts take an unreasonably long time to process, am I holding it wrong? Any models or serving platforms I should try out that might respond reasonably quickly?
Prompt processing is compute bound (raw FLOPS) instead of memory bandwidth bound like token generation. M2 Max is 13TFLOPS. Nvidia 3090 is 35TFLOPS. It’s just the Mac’s GPU being small.
Various suggestions to try things like flash attention which I clearly need to learn more about.
Very cool, but one thing strikes me when typing on actual old hardware.
Responsiveness.
I think its part of what made me enjoy computers in the first place, the immediacy of the commadore printing to screen, then how slow it was to “think” after giving it commands.
I wish there was a way to emulate that on modern hardware.
I recently restored a VT100 out of pure nostalgia (it’s one of the terminals we had in college). If you want me to do any experiments with it, let me know.
What resources did you use? I have a pair of VT240s under my bed I’ve been meaning to get up and running for years but I’ve never found the time and I want to avoid Facebook groups if I can.
The service manuals are available (thank you Bitsavers!) so I referred to that for repair and calibration of the electronics. You can still get parts on eBay pretty easily for the common models. I got a spare VT100 digital board just in case, because DEC used a few custom ICs that are unobtainable. Making the keyboard work better turns out to be the hardest part for me.
BTW, I noticed there’s a connector on the digital board for the 20mA current loop adapter, which has 5V and serial in/out on it. I made a little adapter so I can plug an RPi Pico W into it to run SIMH on, and the back cover completely hides it. It’s like a little standalone time machine.
Ooh, nice! I had been thinking of using a serial multiplexer (either a period appropriate hardware one, or one in software running on an SBC) to connect to multiple things from the same VT240 terminal.
I think it’d be fun to have an ancient terminal sitting next to (and potentially connected to) my modern desktop
Yeah, the Pico can do that too, of course! The idea (not yet implemented) was you would turn on the terminal and get a menu of what to connect to, whether a local emulation or a remote system, much like the terminal concentrator menu we had at CMU in the mid-80s.
I’ve also used the VT100 with my RC2014 motherboard (Z80 running MS BASIC), but switching between that and the Pico involves taking out four screws, so the multiplexer function would be useful there too!
I enjoyed my time with my VT420 as my main interface to my computer, but something has gone wrong with the display, and CRT repairs frighten me, even if I knew enough to have any idea what broke. Not sure whether I want to try to find someone to repair it, or just sell it.
I’m working on the photos, but it turns out you can’t take a good photo of the screen with a phone. At least I can’t. I’m going to dig out the Nikon and try that.
The keyboard is of course the most worn out part of the terminal so it’s hard to say. I’ve had to shave off some plastic from the posts on some keys just to keep them from sticking down. But I can say from examination, and sense memory from my youth, they are just simple springs, there’s no tactile feedback or anything fancy. There is a generated keyclick sound that brought back memories! Maybe I should sample that while I’m at it.
Oooh, this is fun! I’d rather we weren’t burning ridiculous amounts of energy on LLMs, but as long as we are this is at least an easy way to spot them.
Ah. Well, I guess that’s better since that at least leaves room for other service managers to eventually gain those features and a dependency on systemd to be removed.
I wish something like this (at least, that’s still maintained) existed for X11
Did you even write this article yourself?
The writing is all me. While I was writing it I had the idea for that Colophon feature, so I ended up spending half an hour with an LLM to build that tool and create that section of the post.
I experimented with using Claude to suggest improvements for the opening paragraph - if you’re interested here’s the transcript from that: https://claude.ai/share/67f71e7e-8652-429d-96fc-83dc8e3b5453
On a previous post from @simonw, you made a similar snappy comment which later you deleted after others politely replied you. Now you do it again, strangely targetting personally the author. I don’t think this style of communication fits this community.
This is the handout for a workshop I gave this morning at the NICAR data journalism conference. The handout is designed to be useful completely independently of the in-person workshop itself.
I wrote a bit more about this workshop here, including notes on the various new pieces of software I developed in advance of teaching at NICAR.
He wrote the guide for investigative reporters at a data journalism conference.
Yeah, I have no problem at all with the ethics of helping data journalists scrape things that really don’t want to be scraped. News is what somebody does not want you to print.
I originally planned to include notes on running local models for structured data extraction - but I ran into a problem with length limits. For data extraction you need to be able to input quite a lot of text, and the current batch of local model solutions (at least that I’ve tried) tend to stop working well once you go above a few thousand input tokens. Meanwhile OpenAI handles 100,000 and Gemini handles 1-2 million.
In journalism you’re often working with public data, in which case I see no problem at all feeding it into models.
I had some interesting conversations at the conference about the much larger challenge of analyzing private leaks. Those are cases where dumping them into an online model isn’t acceptable - the best solution right now may be investing in a new $10,000 Mac Studio and running capable long context models on that.
What are the most capable long context models nowadays that you can run locally? And why do you specifically need a Mac Studio for that — I’m assuming for the ludicrous amounts of shared memory? How much exactly of it would one need?
That’s something I’ve been trying to figure out. The problem is that longer context requires a LOT of both processing and memory.
I ran a survey about this yesterday on Twitter and on Bluesky - in both cases I asked:
Quite a few interesting answers, including this one:
Various suggestions to try things like flash attention which I clearly need to learn more about.
What is Ibuprofen in this context? It’s completely unsearchable.
the obvious one: painkillers, against the headache from playing
Very cool, but one thing strikes me when typing on actual old hardware.
Responsiveness.
I think its part of what made me enjoy computers in the first place, the immediacy of the commadore printing to screen, then how slow it was to “think” after giving it commands.
I wish there was a way to emulate that on modern hardware.
Nowadays, typing into a terminal emulator is similarly fast, but so is the response! And the printing/display of that response!
Many times a simple command will finish parsing, executing, and printing it’s output before your finger has even left the button
I recently restored a VT100 out of pure nostalgia (it’s one of the terminals we had in college). If you want me to do any experiments with it, let me know.
What resources did you use? I have a pair of VT240s under my bed I’ve been meaning to get up and running for years but I’ve never found the time and I want to avoid Facebook groups if I can.
The service manuals are available (thank you Bitsavers!) so I referred to that for repair and calibration of the electronics. You can still get parts on eBay pretty easily for the common models. I got a spare VT100 digital board just in case, because DEC used a few custom ICs that are unobtainable. Making the keyboard work better turns out to be the hardest part for me.
BTW, I noticed there’s a connector on the digital board for the 20mA current loop adapter, which has 5V and serial in/out on it. I made a little adapter so I can plug an RPi Pico W into it to run SIMH on, and the back cover completely hides it. It’s like a little standalone time machine.
Ooh, nice! I had been thinking of using a serial multiplexer (either a period appropriate hardware one, or one in software running on an SBC) to connect to multiple things from the same VT240 terminal.
I think it’d be fun to have an ancient terminal sitting next to (and potentially connected to) my modern desktop
Yeah, the Pico can do that too, of course! The idea (not yet implemented) was you would turn on the terminal and get a menu of what to connect to, whether a local emulation or a remote system, much like the terminal concentrator menu we had at CMU in the mid-80s.
I’ve also used the VT100 with my RC2014 motherboard (Z80 running MS BASIC), but switching between that and the Pico involves taking out four screws, so the multiplexer function would be useful there too!
I also just think it would be cool to try and use a real terminal for productivity as an exercise
I’m 21 so I’ve only ever used terminal emulators.
I enjoyed my time with my VT420 as my main interface to my computer, but something has gone wrong with the display, and CRT repairs frighten me, even if I knew enough to have any idea what broke. Not sure whether I want to try to find someone to repair it, or just sell it.
Some high resolution photos or video would be nice. And how does the keyboard feel (compared e.g. to common MX switches)?
I’m working on the photos, but it turns out you can’t take a good photo of the screen with a phone. At least I can’t. I’m going to dig out the Nikon and try that.
The keyboard is of course the most worn out part of the terminal so it’s hard to say. I’ve had to shave off some plastic from the posts on some keys just to keep them from sticking down. But I can say from examination, and sense memory from my youth, they are just simple springs, there’s no tactile feedback or anything fancy. There is a generated keyclick sound that brought back memories! Maybe I should sample that while I’m at it.
Sweet baby Jesus on a heavily modified 1000cc motorcycle, this is the most horrifying IoT story I’ve heard in years.
I think the real lesson here is never put your health on the line for a job.
Oooh, this is fun! I’d rather we weren’t burning ridiculous amounts of energy on LLMs, but as long as we are this is at least an easy way to spot them.
They didn’t even hide the SSID
Ooh, cool! Kind of makes me want to experiment with extremely small code size/extremely simple windowing systems like NanoX
Did you know that the postmarketOS folks are also working to upstream musl support for systemd?
Perhaps there should be some collaboration there…
https://postmarketos.org/blog/2024/05/26/the-road-to-systemd
Oh right, that’s because y’all were running into issues with programs having unexpected dependencies on systemd, right?
I was disappointed that you’re having to add systemd to pmOS, but I can’t argue with the maintenance burden of not doing that
Not quite. There are some important features (like managing user services) that are solved with systemd. We wrote all about it here: https://postmarketos.org/blog/2024/05/26/the-road-to-systemd/
Programs/UIs depending on systemd is mostly just a consquence of having many useful things already done in systemd.
Ah. Well, I guess that’s better since that at least leaves room for other service managers to eventually gain those features and a dependency on systemd to be removed.