Working through SICP for the first time and learning Racket along the way. I started last week and have been surprised how much I’m enjoying it. I’ve never used a lisp before and have only dabbled with functional programming in the past.
Getting ready to do Advent of Code in Lua, using my new environment. It’s not really intended to be like an IDE, but it needs all the features anyway, and this seems like a good way to force myself to find all the kinks before someone else does.
Thanks! I’ve been introspecting for most of 2021 on where I wanted to go with Mu. I’d never had any expectation that it would ever be mainstream popular, but I had hoped to coalesce a small community around it, of the scale of say suckless. In 2021, after 5 years of prototypes I found myself asking myself the hard question: why are so few people making the leap from reading about it and starring/liking/boosting/upvoting it to actually trying it out and playing with it? Which led me to the question of, what should attract them to build things atop Mu? The tough answer to face up to was: nothing. Mu today can’t really do much. In particular, not having network drivers is a killing limitation. All you can build with it are toys, and I didn’t set out to build toys.
Though I do have an older variant of Mu that runs on a Linux kernel. That should in principle provide easy network access. I always felt ambivalent about relying on the kernel, though. What’s the point of being so draconian about avoiding C in Mu and building everything from scratch, if my stack includes C from the kernel? And then I learned about how the firmware sausage was made. I’d built Mu to advocate against the constant push to build languages atop other languages, but I started to realize that complexity grows not just upward but also downward. I’d built Mu with some idea of minimizing total complexity of a computing stack, but along the way I started to realize that reality has inextricable complexity that’s independent of what we’re trying to use it for. The goal shifted from minimizing total complexity to “finding a narrow waist of reality,” something that provides a consistent experience above it while relying on very little below it. In this new framing it stopped to matter how deep things extended below. Because really, we’re always going to depend on a deep stack going all the way down to electrons and quarks.
Ok, so networking is hard to recreate and some C is ok but not too much. I started casting about for a minimal stack that’s built directly in C (because minimizing layers of abstraction is still just good engineering). Lua fits the bill quite nicely. Linux kernel + libc + 12kLoC for Lua is arguably as minimal an implementation of networking as I’m going to get. And Lua packs an impressive amount of technical depth in 12kLoC to make its programs go fast. Graphics is still too much, though. The Lua eco-system already has https://love2d.org for anyone wanting to do graphics. I have nothing to offer there. But I hope to make progress on the original goal of Mu, before all the feature creep:
Apps are easy for anyone to run. The infrastructure they need is easy to build.
When modifying an app, it’s difficult to end up in irrecoverable situations or bugs that are too hard to track down. (That’s really what all of Mu’s memory safety was aiming for.)
So I’m going to spend the next 5 years wondering why nobody’s trying this out :D It’s quite possible that text mode is just a bridge too far for everyone else. We’ll see.
To compare Teliva against what seems like obvious competition:
TIC-80 & Pico-8:
Why would I go with Telvia vs either of those? Networking is a partial reason, but if I’m going to write networked software, I am likely taking on sufficient complexity that I also have access to a less constrained interpreter elsewhere.
Janet:
Janet is a rather nice language, and has somewhat of a head start on getting the large amount of cross platform work to do things deeper in the stack. Granted, Janet is a harder sell than Lua for a small language, and it doesn’t enable bundling source in the same deep way, but in an age where git is everywhere, that can feel like less of a big deal?
Further thoughts
So, like, now that the bottom of the software in question isn’t in scope, it strikes me that you’re in much more of a marketing project now, as opposed to the heavily technical project that Mu was.
I can see some benefits to Teliva, especially if you build it to have an in-browser sandbox, and builds that work on windows as well (at least, if you want this to have penetration outside of the macOS/Linux world). Other things that I think would be handy would be widgets, and support for inline “picture” variables that represent escape-code-based strings (or sequences of curses commands).
To be honest, it kinda reminds me of what QBasic was, in a way. Taking advantage of that slim middle and making sure that Teliva runs everywhere is what I see as a potential attraction point, along with the source editing.
Also, for useful apps, many of them will need some sort of persistent state, unless they are strictly API connected. Do you have any thoughts there? (Persistent “variables” come to mind for me?)
Do you plan on trying to make it easy to share .tlv files? If you can make them easy to host on itch.io (via some base interpreter compiled to WASM a la Love2d), you’ve got a distribution mechanism full of hobbyists that are into this sort of thing, and could have it used in Game Jams (which some people would definitely go for, retro looks are popular for those).
Oh, I totally forgot to mention the most important property of Teliva: sandboxing. It’s kinda implied by the idea of making apps easy to run. I want to be able to easily share my apps with others, and to be able to run apps from others without having to audit their code. There’s 0 sandboxing at the moment, but that’s going to be 90% of the work from here on out. And that’s the big difference with existing HLLs. What we call programming language runtimes today don’t provide any sandboxing primitives. Why not?
You called the browser a sandbox above, so clearly you were thinking along similar directions :) I consider the browser to be a great example of a failed sandbox. Failed because I have zero confidence in the security of my computer today, and the browser is a big part of that experience. Even though browsers were designed with a well-defined sandboxing model, that fundamental model is obsolete. We used to consider the hard disk the crown jewels to be protected, and websites to be throwaway. The situation is almost entirely flipped today. Particularly if you’re on a Chromebook. Browsers try to keep up by patching that fundamental model, but at huge cost in implementation complexity. When I visit a website today I basically have no good sense of what that website can and cannot do (except that it cannot install software on my computer). Only recently did I realize that when I allow a website to send me notifications, it can do so even after I close my tab.
The one segment that is perfectly sandboxed today is fantasy consoles like TIC-80 and PICO-8. But they’re sandboxed by the easy expedient of just not being able to do much. Screen, keyboard, mouse/touchpad, that’s about it.
So, to answer your question, I want Teliva to be:
safer for non-technical people to run untrusted programs on than any HLL in existence
as fun as fantasy consoles in opening up untrusted programs and poking around at their internals
more capable than fantasy consoles in terms of the kinds of apps that can be built (access to local disk, facebook API, etc.)
easier to use and freer of footguns than any shell, with a well-defined set of things users might conceivably need to know about. (I have a local alias called stow. I recently discovered I couldn’t hit tab to autocomplete files after typing stow. Turns out there’s a GNU tool called stow that isn’t installed on my system. But zsh does install autocomplete rules for it.)
I started writing my site as a static site generator in Rust mostly just because I love practicing Rust and I’d like to be more fluent in it.
I’ve also been moving more of my free time to making music/art (generative art included). I finally got myself a DAW for Black Friday that I’m going to try to run some tutorials with this week.
I’m still pet sitting at a friend’s house up in the mountains.
And while I do that, I’m reading a book on “farmhouse brewing” and looking for interesting things to do with hexagon grids and OpenGL.
Unfortunately, a good friend of mine back in Boulder tested positive for covid over the weekend, and since we were hanging out all last week, I’m basically just isolating in the house. At least I get to play with cats here.
I got tested this morning, and the rapid scan test came up negative, but of course I’m still waiting for the PCR results. I feel fine except for a slight sore throat, but it could be from riding in the cold air here. My friend is miserable, but since she was vaccinated and able to get a monoclonal antibody IV over the weekend, hopefully it will turn around soon.
$WORK: Getting our stack off of MongoDB permanently, one step at a time.
$HOME:
2 weeks after posting here that my laptop was barely hanging on, the battery just….stopped being recognized entirely. Ended up pulling the trigger on an M1 Macbook Air and that arrives tomorrow, so playing with that will constitute most of my week.
Playing around with modeling some custom enclosures for a couple electronics builds
Nice and normal MySQL haha. The Mongo was leftover and horrible from back when the company was founded 7y ago and it was not managed properly, plus our data is headed far more in the relational direction now.
I finished writing a program to capture packets off the network and store them in a proprietary format that includes embedded indexing info. I’m now writing a program to convert our proprietary capture file format to the pcap file format.
Once that’s finished, I need to make some changes to our Suricata module that feeds the packets from our proprietary capture format into Suricata. We call this feature “RetroIDS” since you can rescan old enriched captures with new signatures.
We’re almost ready to finally ship our completely redesigned product. When I joined the company in May 2020, I saw that they had some really cool tech on their hands, but it wasn’t unified. It was a set of disparate applications that seemed to work alright. I sought and got approval to redesign the entirety of the tech in a unified way. This was such a disruptive change that it caused us to even restructure the company.
And now we’re about to launch. This week is going to prove crucial in preparation for that launch. We have a lot resting on this particular week.
With HardenedBSD:
On Thanksgiving Day (US holiday) last week, I installed my home’s first rack (an open 25U rack). HardenedBSD’s second ThunderX1 system arrived as well. I’m working to bisect a FreeBSD commit that triggers a kernel panic. I’ve almost got it narrowed down.
I also need to write the HardenedBSD November 2021 Status Report.
$WORK: Attempting to build some status reporting and data management tools we’ve needed for like two years. Needs to be in at least a vaguely-shippable state by next week, so, yeah.
$HOME: Cleaning things up, both physically and metaphysically. House, but also finances, old chores, etc.
Finishing my own esp32 board: usable for prototyping but also usable in production (for personal projects).
It’s like a typical devboard but no on-board USB-TTL converter (they’re comparatively expensive, use board space with many additional components, and use precious power), some pins laid out several times in order to make connecting to SPI and I2C buses easier, and a battery connection but no on-board charge circuit.
I’m remotely attending the second Dagstuhl seminar on secure compilation. Lots of interesting talks lined up. It’s a hybrid format with about 2/3 of the attendees remote. We’ll see how well Gathertown replaces the informal discussions that usually make these things so valuable. In-person, the fact that Dagstuhl has a good selection of wine and beer available at cost prices is very good for facilitating good discussions in the evenings.
It’s great to see that there is still some way to have a Dagstuhl seminar, as they are so inspiring and fun. Even though it’s hybrid, I’m sure it will be enjoyable :)
The first one in this series was amazing (though I made the mistake of complaining that the balance between theoretical and applied folks was a bit uneven and so got roped into co-organising this one). This one was meant to happen last spring. Dagstuhl was very late in cancelling it - our policy was no work travel allowed and Dagstuhl was still very hopeful that the seminar would take place until a couple of weeks before. I’m a bit surprised that they let this one go ahead - Germany was starting to hit the 4th wave a month ago, so it was pretty obvious that a lot of folks wouldn’t be able to make it. Luxembourg closed its borders to non-EU folks, which made getting to the venue harder even for the folks who were willing to travel (LUX is the closest airport).
They’ve done an amazing job in Gathertown, recreating the house. Unfortunately, they’re using Zoom for the main events and the switching between Zoom and Gathertown is not great (Gathertown can only handle sharing screen to 100 people, but for an event with 45 attendees that would be fine).
I’m going to be working on a weekly newsletter showcasing computer ads from days gone by. I also have to work on an article about this history of Unix from v1 to the creation of BSD.
I’m spending the week at AWS re:Invent. I’m presenting two chalk talks about Bottlerocket and spending the rest of the time around the conference to talk to folks about Bottlerocket and containers in general.
We still don’t have water back at our existing (rented) house after Storm Arwen hit last Friday, but today we got the keys to our new house (15 minutes away), and we have water there. Looking forward to a proper shower! We’re expecting to move “for realz” in two weeks time—until then we’ll paint internal ceilings and walls, and clean, interspaced by $DAY_JOB.
Working through SICP for the first time and learning Racket along the way. I started last week and have been surprised how much I’m enjoying it. I’ve never used a lisp before and have only dabbled with functional programming in the past.
After about 50 days of break, will start working on a book again. I’m planning to revamp Vim reference guide that I wrote a few years back.
Getting ready to do Advent of Code in Lua, using my new environment. It’s not really intended to be like an IDE, but it needs all the features anyway, and this seems like a good way to force myself to find all the kinks before someone else does.
That sounds neat/cool! I have other advent coding plans around a weekly art project that will mostly likely be p5 based.
I’ve been out of touch, how have you ended up on Lua for this environment?
Thanks! I’ve been introspecting for most of 2021 on where I wanted to go with Mu. I’d never had any expectation that it would ever be mainstream popular, but I had hoped to coalesce a small community around it, of the scale of say suckless. In 2021, after 5 years of prototypes I found myself asking myself the hard question: why are so few people making the leap from reading about it and starring/liking/boosting/upvoting it to actually trying it out and playing with it? Which led me to the question of, what should attract them to build things atop Mu? The tough answer to face up to was: nothing. Mu today can’t really do much. In particular, not having network drivers is a killing limitation. All you can build with it are toys, and I didn’t set out to build toys.
Though I do have an older variant of Mu that runs on a Linux kernel. That should in principle provide easy network access. I always felt ambivalent about relying on the kernel, though. What’s the point of being so draconian about avoiding C in Mu and building everything from scratch, if my stack includes C from the kernel? And then I learned about how the firmware sausage was made. I’d built Mu to advocate against the constant push to build languages atop other languages, but I started to realize that complexity grows not just upward but also downward. I’d built Mu with some idea of minimizing total complexity of a computing stack, but along the way I started to realize that reality has inextricable complexity that’s independent of what we’re trying to use it for. The goal shifted from minimizing total complexity to “finding a narrow waist of reality,” something that provides a consistent experience above it while relying on very little below it. In this new framing it stopped to matter how deep things extended below. Because really, we’re always going to depend on a deep stack going all the way down to electrons and quarks.
Ok, so networking is hard to recreate and some C is ok but not too much. I started casting about for a minimal stack that’s built directly in C (because minimizing layers of abstraction is still just good engineering). Lua fits the bill quite nicely. Linux kernel + libc + 12kLoC for Lua is arguably as minimal an implementation of networking as I’m going to get. And Lua packs an impressive amount of technical depth in 12kLoC to make its programs go fast. Graphics is still too much, though. The Lua eco-system already has https://love2d.org for anyone wanting to do graphics. I have nothing to offer there. But I hope to make progress on the original goal of Mu, before all the feature creep:
So I’m going to spend the next 5 years wondering why nobody’s trying this out :D It’s quite possible that text mode is just a bridge too far for everyone else. We’ll see.
To compare Teliva against what seems like obvious competition:
TIC-80 & Pico-8:Why would I go with Telvia vs either of those? Networking is a partial reason, but if I’m going to write networked software, I am likely taking on sufficient complexity that I also have access to a less constrained interpreter elsewhere.
Janet:Janet is a rather nice language, and has somewhat of a head start on getting the large amount of cross platform work to do things deeper in the stack. Granted, Janet is a harder sell than Lua for a small language, and it doesn’t enable bundling source in the same deep way, but in an age where git is everywhere, that can feel like less of a big deal?
Further thoughtsSo, like, now that the bottom of the software in question isn’t in scope, it strikes me that you’re in much more of a marketing project now, as opposed to the heavily technical project that Mu was.
I can see some benefits to Teliva, especially if you build it to have an in-browser sandbox, and builds that work on windows as well (at least, if you want this to have penetration outside of the macOS/Linux world). Other things that I think would be handy would be widgets, and support for inline “picture” variables that represent escape-code-based strings (or sequences of curses commands).
To be honest, it kinda reminds me of what QBasic was, in a way. Taking advantage of that slim middle and making sure that Teliva runs everywhere is what I see as a potential attraction point, along with the source editing.
Also, for useful apps, many of them will need some sort of persistent state, unless they are strictly API connected. Do you have any thoughts there? (Persistent “variables” come to mind for me?)
Do you plan on trying to make it easy to share .tlv files? If you can make them easy to host on itch.io (via some base interpreter compiled to WASM a la Love2d), you’ve got a distribution mechanism full of hobbyists that are into this sort of thing, and could have it used in Game Jams (which some people would definitely go for, retro looks are popular for those).
Oh, I totally forgot to mention the most important property of Teliva: sandboxing. It’s kinda implied by the idea of making apps easy to run. I want to be able to easily share my apps with others, and to be able to run apps from others without having to audit their code. There’s 0 sandboxing at the moment, but that’s going to be 90% of the work from here on out. And that’s the big difference with existing HLLs. What we call programming language runtimes today don’t provide any sandboxing primitives. Why not?
You called the browser a sandbox above, so clearly you were thinking along similar directions :) I consider the browser to be a great example of a failed sandbox. Failed because I have zero confidence in the security of my computer today, and the browser is a big part of that experience. Even though browsers were designed with a well-defined sandboxing model, that fundamental model is obsolete. We used to consider the hard disk the crown jewels to be protected, and websites to be throwaway. The situation is almost entirely flipped today. Particularly if you’re on a Chromebook. Browsers try to keep up by patching that fundamental model, but at huge cost in implementation complexity. When I visit a website today I basically have no good sense of what that website can and cannot do (except that it cannot install software on my computer). Only recently did I realize that when I allow a website to send me notifications, it can do so even after I close my tab.
The one segment that is perfectly sandboxed today is fantasy consoles like TIC-80 and PICO-8. But they’re sandboxed by the easy expedient of just not being able to do much. Screen, keyboard, mouse/touchpad, that’s about it.
So, to answer your question, I want Teliva to be:
stow. I recently discovered I couldn’t hit tab to autocomplete files after typingstow. Turns out there’s a GNU tool calledstowthat isn’t installed on my system. But zsh does install autocomplete rules for it.)In exchange for these, I’m currently setting aside pixel graphics. About the best I can manage is this Game of Life app using Braille characters.
$work: looks like I’m writing a simulator to test a distributed system.
$else: I’m working on my zine/book about bitmap drawing math and algorithms with code examples in Rust, A Bitmapper’s Companion
I started writing my site as a static site generator in Rust mostly just because I love practicing Rust and I’d like to be more fluent in it. I’ve also been moving more of my free time to making music/art (generative art included). I finally got myself a DAW for Black Friday that I’m going to try to run some tutorials with this week.
I’m still pet sitting at a friend’s house up in the mountains.
And while I do that, I’m reading a book on “farmhouse brewing” and looking for interesting things to do with hexagon grids and OpenGL.
Unfortunately, a good friend of mine back in Boulder tested positive for covid over the weekend, and since we were hanging out all last week, I’m basically just isolating in the house. At least I get to play with cats here.
I got tested this morning, and the rapid scan test came up negative, but of course I’m still waiting for the PCR results. I feel fine except for a slight sore throat, but it could be from riding in the cold air here. My friend is miserable, but since she was vaccinated and able to get a monoclonal antibody IV over the weekend, hopefully it will turn around soon.
$WORK: Getting our stack off of MongoDB permanently, one step at a time.
$HOME:
where’s your stack headed, db-wise?
Nice and normal MySQL haha. The Mongo was leftover and horrible from back when the company was founded 7y ago and it was not managed properly, plus our data is headed far more in the relational direction now.
And, mind to share why you’re moving off?
See other comment :)
I finished writing a program to capture packets off the network and store them in a proprietary format that includes embedded indexing info. I’m now writing a program to convert our proprietary capture file format to the pcap file format.
Once that’s finished, I need to make some changes to our Suricata module that feeds the packets from our proprietary capture format into Suricata. We call this feature “RetroIDS” since you can rescan old enriched captures with new signatures.
We’re almost ready to finally ship our completely redesigned product. When I joined the company in May 2020, I saw that they had some really cool tech on their hands, but it wasn’t unified. It was a set of disparate applications that seemed to work alright. I sought and got approval to redesign the entirety of the tech in a unified way. This was such a disruptive change that it caused us to even restructure the company.
And now we’re about to launch. This week is going to prove crucial in preparation for that launch. We have a lot resting on this particular week.
With HardenedBSD:On Thanksgiving Day (US holiday) last week, I installed my home’s first rack (an open 25U rack). HardenedBSD’s second ThunderX1 system arrived as well. I’m working to bisect a FreeBSD commit that triggers a kernel panic. I’ve almost got it narrowed down.
I also need to write the HardenedBSD November 2021 Status Report.
$WORK: Attempting to build some status reporting and data management tools we’ve needed for like two years. Needs to be in at least a vaguely-shippable state by next week, so, yeah.
$HOME: Cleaning things up, both physically and metaphysically. House, but also finances, old chores, etc.
Finishing my own esp32 board: usable for prototyping but also usable in production (for personal projects).
It’s like a typical devboard but no on-board USB-TTL converter (they’re comparatively expensive, use board space with many additional components, and use precious power), some pins laid out several times in order to make connecting to SPI and I2C buses easier, and a battery connection but no on-board charge circuit.
I’m remotely attending the second Dagstuhl seminar on secure compilation. Lots of interesting talks lined up. It’s a hybrid format with about 2/3 of the attendees remote. We’ll see how well Gathertown replaces the informal discussions that usually make these things so valuable. In-person, the fact that Dagstuhl has a good selection of wine and beer available at cost prices is very good for facilitating good discussions in the evenings.
It’s great to see that there is still some way to have a Dagstuhl seminar, as they are so inspiring and fun. Even though it’s hybrid, I’m sure it will be enjoyable :)
The first one in this series was amazing (though I made the mistake of complaining that the balance between theoretical and applied folks was a bit uneven and so got roped into co-organising this one). This one was meant to happen last spring. Dagstuhl was very late in cancelling it - our policy was no work travel allowed and Dagstuhl was still very hopeful that the seminar would take place until a couple of weeks before. I’m a bit surprised that they let this one go ahead - Germany was starting to hit the 4th wave a month ago, so it was pretty obvious that a lot of folks wouldn’t be able to make it. Luxembourg closed its borders to non-EU folks, which made getting to the venue harder even for the folks who were willing to travel (LUX is the closest airport).
They’ve done an amazing job in Gathertown, recreating the house. Unfortunately, they’re using Zoom for the main events and the switching between Zoom and Gathertown is not great (Gathertown can only handle sharing screen to 100 people, but for an event with 45 attendees that would be fine).
I hope I’ll get to more of them in the future.
$WORK: payment integrations
$HOME: webworkers and offscreen canvases.
I’m going to be working on a weekly newsletter showcasing computer ads from days gone by. I also have to work on an article about this history of Unix from v1 to the creation of BSD.
For those interested, over the weekend I published the second entry in my newsletter about cool historical tidbits: https://historicaltidbits.substack.com/p/britains-frozen-aircraft-carrier.
This sounds very cool!
I’m spending the week at AWS re:Invent. I’m presenting two chalk talks about Bottlerocket and spending the rest of the time around the conference to talk to folks about Bottlerocket and containers in general.
$WORK: Planning work for the next year with the team.
$HOME: Trying to parse AML in rust to get PCI configuration for a rust kernel.
pulling out the relevant pieces of https://eisman.github.io/neo4jd3/ for my own application.
We still don’t have water back at our existing (rented) house after Storm Arwen hit last Friday, but today we got the keys to our new house (15 minutes away), and we have water there. Looking forward to a proper shower! We’re expecting to move “for realz” in two weeks time—until then we’ll paint internal ceilings and walls, and clean, interspaced by $DAY_JOB.
Writing the last few bits of a scripted secret distribution engine. :)
I continue to debug my function inliner. Hopefully fewer crashes this week.