I’ve been becalmed for several weeks on my language that skips Assembly and C and translates directly to machine code (paper [pdf; 25 pages]; repo; blog post).
The problem: I’ve been using unsafe addresses all over the place even though I always planned to switch them to fat pointers which protect against accidental memory corruption. It was just easier to put off that concern while I got things off the ground. But now I realize that fat pointers don’t fit in registers (which my language exposes) and so can’t be returned from functions (because functions can only return registers). So I’m having to go through and rewrite a bunch of function signatures and call-sites, while moving local variables all over the place from registers to the stack. All while programming in machine code and having to adjust struct.field offsets by hand. It’s stressful work, because I need to fix things everywhere in a program before I can run any of its tests. I know that if I’m not careful up front I’m liable to end up in a hole I can’t debug myself out of.
The upshot: if you look at my repo there isn’t a lot of recent commits. But that’s because the work has been happening on a private branch. Most of the repo has been translated, with the exception of the compiler from the HLL to machine code, the biggest program written in Mu by far. There I’m about 40% of the way through the translation, but none of the 40% has been actually verified, so who knows.
Being forced to stay home isn’t helping. My presentation for the conference next week (free to join!) is also taking away some momentum. But that’s actually going well.
I’m going to just keep at it. It’ll take as long as it takes. And hopefully I won’t need to do stuff like this again for several years at least.
Hi akkartik, I recall interacting with you once on HN about one of your languages. I’m struck by how off the beaten path the kind of work that is. If you don’t mind me asking, do you have a long-term vision/goal with your language projects, or is it something you are just doing out of interest? I can sympathize either way.
Hope you are well.
edit: I see your blog gets into this. Very cool work. We need more radical experiments like this.
Why thank you! Until about 2015 or so I would periodically lose interest in my side project and cast about until I found another. Since then, though, I’ve been on a single track, so I feel very fortunate.
I found our old conversation in my records. Good to see you here, and I hope you are well too. Feel free to email or DM me if you have more questions.
When into bootstrapping, my first goal was to get above tedious manipulations. Learning from Lisps and meta languages, the idea was to automatically bake in some ability to do metas that let me write higher-level functions that converted to lower-level. Maybe automate things like offsets, too. If too primitive and close to machine, then the developer has to essentially think like a machine on a regular basis for improvements. It just seemed like we as human beings shouldn’t be doing that. Some people like it, though.
I did think I’d want to see, REPL-style, what each high-level statement generated. That way I could keep stuff efficient and/or secure. Also good to be able to just inline some hand-written ASM or machine code if I’d want to do it better.
My personal goal was the minimum number of steps I could do to immediately get away from ASM. The design called for an interpreter hand-written in minimal ASM or machine language supporting basic expressions, structs, function calls, arrays (for file I/O), basic I/O ops, linked lists, trees, some operators on them, and substitution-based metaprogramming. All aren’t strictly necessary but are convenient. With that in place, the rest can be layered on top with the interpreter executing or, if output is logged, used as a pseudo-compiler. The simplified tools bootstrappers collected, such as for ELF’s, could be ported to that foundation to run similarly. Dropped the second we get TCC, LCC, or whatever through it. I’d have automated the TCC port using existing tooling for C transforms followed by distributed eyeballing and equivalence testing.
That’s where I left it. I enjoy seeing the solution more than tedious work of building it. Impressive that you’re still keeping yours low-level despite the challenges involved.
“ because I need to fix things everywhere in a program before I can run any of its tests. “
Quick note to anyone in this space. Another advantage of an interpreter with meta-programming is that you can build and test each feature in isolation. Then, you can test them together. The meta can support that with some code reuse by having two modes, running and testing. Kind of like ifdef’s in C codebases but integrated with language. Even bootstrappers should be able to enjoy productivity boosts.
Although, it really depends on one’s goal and associated constraints. Mine was just defeating Karger-Thompson attack on C toolchains in widely-verifiable way. Some like akkartik want deep understanding and control of low-level details from bottom to top. I still think they could use extra features if only to be like having IDE support saving time on tedious stuff.
Yeah, Mu could certainly use extra features over machine code. We already have struct offsets. Try this out on a Linux machine:
$ git clone https://github.com/akkartik/mu
$ cd mu
$ cat > z.mu
type point {
x: int
y: int
}
fn main -> exit-status/ebx: int {
var p: point
var a/ecx: (addr int) <- get p, x
copy-to *a, 3
var b/edx: (addr int) <- get p, x
exit-status <- copy *b
}
^D
$ ./translate_mu z.mu
# generates a.elf
$ ./a.elf
$ echo $?
3
In your language, can functions return results in more than one register? Is there a way to express that a single two-word value should be stored in two registers?
can functions return results in more than one register?
Yes. Functions can return results in as many registers as they want. The registers of the results are part of the signature just like the types.
The calling convention is for the callee to preserve all registers except those that return results.
Is there a way to express that a single two-word value should be stored in two registers?
No, there’s no way to manually perform scalar replacement at the moment. I don’t yet know how to preserve type-safety in that situation. It would likely require some additional syntax, and it’s not obvious that the benefits outweigh the additional complexity.
I know the feeling. My regex compiler has the same issue. If I pass invoke the wrong methods in asm, I often just get a null pointer in the library. The next step is a bytecode dump that can’t be parsed by the standard tools. And in a compiler (or at least a compiler written by me…), you can have to change far too many entangled things before you get a useful result.
The upshot is that I’ve ended up twenty commits deep in a branch, slowly working to just get something that produces runnable code. Eventually I get there, and I can start to worry about the mundane details, like “do my tests pass?”
I also want to prototype my first ActivityPub app: a PWA (progressive web app) focused exclusively on posting authoring. Something to help me focus on creative work without getting distracted by new posts in my feed.
How do you get started with The Domino Project? It seems like a cool project, but it seems like all the information is spread out in different “blog posts”?
I wish I could recall but it was years ago. If I had to guess, some combination of blog posts and podcasts bringing the individual books to my attention.
I wrote a toy webserver that listens on a port and responds to only one request before moving to a new port. It also tells you where it’s going to migrate to :D
So far it’s deployed as a capture the flag type exercise, but I might wrap some service discovery systems around it in order to learn more about that domain and the tools available.
Interesting! I wonder how seamless you could make the experience be using JavaScript. Another idea I just had would be to make it into a choose-your-own-adventure game: you’d make a choice by going to a different port. I’m not sure if that would be any more fun than just clicking a link, but it could be a fun demo.
Yep, a client that implements that seamless-ness is required for finding the flag in the current version - in addition to the next port it will listen on, the server also provides a single, random, indexed value from the flag. In order to solve the challenge you would need to write some code that follows the breadcrumb trail for long enough to suss out the flag.
Glad you found it interesting, it was a lot of fun to write. Python’s aiohttp is super neat.
Hopefully finalizing the write-heavy, heavily-shared object store for the next generation of our product. It’s complicated because it’s not just a flat database; it’s closer to a strongly typed graph database but for reasons it is built on top of a classical relational model in PostgreSQL.
Every graph-oriented database I experimented with didn’t have one or more of the following required features:
handling of huge numbers of writes a second (most tend to be optimized for read-heavy, and/or just don’t allow a lot of simultaneous writes)
good compound (multi-attribute) indexing (a lot have indexes, of course, but they tend to not handle range queries well or require some concept of a single-attribute primary key)
handling of ~240 of simultaneous connections (some seem to be designed as a collaboration tool rather than an enterprise database)
good scaling on a single device (a lot of them seem to have the only real scaling mechanism be spreading over multiple devices)
good handling of IP addresses as first-class types (this basically doesn’t exist)
good handling of resource-exhaustion cases (a lot of them have no problem just dying on out-of-memory or write failure or whatever)
If any crustaceans know of any graph-oriented DB that meets these sort of requirements, I’d love to hear about it, thank you.
Over the weekend I experimented with automatic code generation. We have a large, but pre-defined, set of object classes to be stored, which is amenable to code generation but…the predefined list isn’t expected to change all that much, so it boils down to the classic problem of spending a lot of time to build the tool and save time later or not.
On the personal front, I started a new book series with each of my kids. On nice days, we sit outside and I read a chapter to each of them out of each of the book series we’re doing (we also do a chapter every night at bedtime). The older boy and I just started on Percy Jackson and the younger boy and I are doing The Last Firehawk series. We’re making good progress through those books.
(If you have a preschooler who really wants to like Harry Potter like their older sibling(s) but doesn’t have the staying power to make it through long chapters and complex plots, The Last Firehawk might be for you.)
There’s a big community for randomizers.
You reverse engineer an old game, put the items in different spots, and people
organize races on the resulting game, having to route on the fly where to go
next depending on what they know they can do (in-logic), what they know they
can do with glitches (out of logic), and what they can’t do (item locked).
(I should write a full-length piece on this, fascinating stuff and display of
technical skills.)
As a speedrun and rando enthusiast, I follow a few runners, specifically The
Legend of Zelda: Ocarina of Time Randomizer runners.
One of them (Marco) said that if there was a ladder (ELO-style ranks for 1v1
races) for OoT he’d do more rando.
I’m out of a job, I can’t get out of my apartment, so I went and did it:
Kaepora is born. A race planner with
Glicko-2 leaderboards and a Discord bot.
The first public race finished as I wrote this, it’s a success.
16 people were competiting this evening just for the sake of running OoTR and
knowing who’s the fastest. Absurd seeds were had, good and bad routing
decisions were made, some were lucky, and some had their first bomb bag on 50
skulls.
This is already peak niche nerd stuff, but next I’ll open the randomized
settings randomizer league. Twice the rando, twice the fun.
I have an extense to-do list that increases its content every day, so my weekend is going to be to try and make a lot of tasks, give my server a bit of maintenance, and more time for my projects.
Also, writing a ton of drafts for my blog so I can keep publishing stuff.
I’ve got to try to figure out how to focus, mainly for work, but also for some personal stuff. Between isolation and being three months into a new job, I just can’t sit and focus on my work. Not “oh man after three hours of meetings in a day, I’m only coding five more hours”; more like “I’m spending six hours in front of the computer and not even being productive for 30 minutes total”.
I’m working on building an electronic chess board, apparently it’s not a new thing but it’s a super excited project to work on. I documented the process on my personal blog.
@home: Prompted by @hwayne’s request for “FactoryBot” style APIs for property based testing, I’ve tried my hand at it. Over the weekend, I created a pretty small POC. I actually ended up spending more time on stereotyped tests than generators, but I have ideas for both sides of it.
Short version, the idea is that you can write things like:
assertTrue(operator((Integer x) -> x % 2, Integer.class).isIdempotent());
I have a bunch of ideas for improving the syntax for declaring variables, and there’s also a lot of properties that you might want to check for functions. Beyond that, anyone who looks at the code will realize I don’t know much about Choco Solver, which I’m using under the hood, and I need to learn more about that.
Continuing to make progress on my personal site. Deployed it with Phoenix Liveview yesterday, really no reason for the Liveview other than the fact that it’s fun :)
I’m also doing chapter 2 and 3 of the https://grox.io Elixir course.
$WORK: It’s paper deadline time again! We’re submitting one paper on Thursday, so this week is all about wrapping that up. Then the next day, I have a final project presentation for a class before doing the last sprint on another paper due early May. After that, I start my virtual summer internship.
Quarantine has been getting to me more lately, so I’ve been resolving to try to have more of a schedule and get more time outside in the sun. I’m finding it pretty hard to stay motivated sometimes, but having this work schedule with the papers helps a lot, surprisingly.
$HOME: I had a lovely anniversary weekend with my wife, and I really enjoyed having time completely away from work, so I think I’ll keep that up. That said, I’ve been reading a professor’s blog and am feeling very motivated to do more writing on my blog and I’d love to start that up again. I felt very accomplished after finishing the last post I wrote up.
Other than that, it’s just going to be more crime drama bingeathons and relaxing.
This week I want to start an art project. I either want to do an illustration project or a music project with my guitar. Maybe even both. I’m not sure what I want to do yet though. Anyone have any recommendations for someone who’s learning illustration and/or guitar?
$HOME: Finally got my laptop back from HP, who repaired my laptop hinges for free. I backed up/wiped nearly everything before I sent it in, so I set it up to now dual-boot Windows 10 and Linux. I’m wrapping up lessons learned from “No code after work”, so I’m ramping work back up on my 2D C99 game engine.
$WORK: I’ve been struggling for quite a few months, but putting in the effort so things are slowly turning around.
Working on some iOS client work. I’ve decided I should really become fluent in Swift so I’ve been working my way through the swift programming language book, and doing some advent code along side. I kind of want to start building an NES emulator! Seems like fun.
I can’t help but wonder when I’ll just start putting more time into Rust. But for now, I think the payoff is something I need to use as a guide for my decision making.
Also, I just signed up for a write.as pro trial. So I’m going to try and set up my blog, and start writing!
I’ve been timetabling my days to give me some structure so that I don’t waste the longest holiday of my life (I’m meant to be on exam leave, except exams have been cancelled). My day looks something like this:
Organisation. I’ve been going through all the paper which I’ve kept - pretty much everything I’ve written down since about age ten I’ve kept. Some of it I find surprisingly insightful considering, but mostly it’s interesting to see how much I’ve changed. A lot of it’s being discarded, but I’ve been keeping various things in different forms: illustrations and poetry is going into a scrapbook type thing, notes I deem worthwhile are going into ringbinders by date. Miscellaneous texts, I’m writing up onto the computer; usually I’d prefer not to do this but it is helping me to decide what to keep and what not.
Music. I have quite an extensive digital music collection, but since listening to some old CDs and vinyls again, I’ve realised that I approach the latter two in a very different way, and it’s a way I prefer. As a result I’m spending an hour listening to a couple of albums all the way through every day, with the intention of buying physical copies of those which I enjoy the most. Anything I don’t particularly like I’m deleting forever, which I don’t have a huge problem with seeing as most of it was obtained dubiously (ripped from friends’ CDs, shared by friends, torrents, and Google’s one-pound album deals they used to do).
Reading. I’m taking this opportunity to try and get through my backlog and book to-do list, which will be a challenge as it’s over 200 books! I’m starting with the physical books I have available, and then moving on to classic books I can download for free. Unfortunately I sat on my Kindle, rendering the screen unusable, so I’m also looking for replacement ereaders on Ebay.
Sewing. I’ve always wanted to be able to sew my own clothes. I’m starting out with baby steps from a children’s book, and I’m going to work upwards from their. Thus far I’ve only been sewing on paper, which is an interesting learning technique but is working well for me.
German. I haven’t studied another language with any degree of seriousness since GCSEs, when I really went off the experience after our enthusiastic and passionate teacher left and was replaced by another who was only filling in until the end of the year and made it obvious she didn’t give a shit. I do quite like the German language though, so I want to get back into learning it. Currently I’m watching German beginner videos and using Duolingo; I don’t know how far it’ll take me but I think it’s a good start. If anyone has any recommendations for alternative pathways please let me know!
Those activities usually take me up to about 4, after which I’ll go to the allotment for a bit, and then come home and work through K&R. I think having some kind of a structure is making the whole thing more enjoyable.
I once had a summer (in high school - which goes to 20 year old where I’m from btw) where I decided not to work and just tackle my list of things that I wanted to do.
By the end of the summer I had a firefox session with a few thousand tabs and anxiety about all of the things that I needed to know but didn’t yet. The culprit was that instead of treating the list as a stack or a FIFO queue I tried to make a priority queue, calculating the priority wasn’t decidable and the net effect was just to add more things to the list. I have never had a busier summer, it felt like time was speeding up exponentially.
If I had been able to focus I would maybe have learned a single useful thing instead of an encyclopedia’s worth of trivia that I was unable to manifest (i.e. pointers to more information rather than actual skills).
I think that the only thing that’s actually going to be an issue for is the German, as most of the other things aren’t so much about learning as doing. In previous summers I’ve tried sticking with a single thing for the whole time, but I can’t concentrate like that.
In some ways, I am treating it like a FIFO queue, just with multiple queues running simultaneously. Essentially I want the structure to still be there throughout the day, as having nothing compulsory to fill the day with and this long holiday, before going to study a course without a huge amount of contact time seems like a recipe for dropping out. As some of the activities near completion (“lessons” one and two are nearing the end even now) I can choose another thing from my backlog or increase the amount of time on one of the other tasks.
Thank you for the helpful response and for the concern though. I think that for now I’ll stick with my current system, and if in a couple of weeks I feel like I’m achieving nothing I’ll double down on one and you can say I told you so :)
As good as. I’ve modified the code for various programs before, but that was more guesswork when something didn’t work quite how I wanted, and I’ve followed a few tutorials for C projects, but never fully understood what was going on when the less intuitive features were whipped out. I figured it’d be a much smoother experience and allow me to work on some other things I wanted to work on if I actually understood the language. I’m having quite a lot of fun with it!
Reading some more pages in the analects. Working, looks like a simple sprint. Studying topology (applying for university and was missing topology from my transcipt).
I’m currently funemployed (by choice) and am treating the time until I seriously start looking for a job as ongoing programmer vacation, kinda.
I’m migrating an old bash templating theme system to babashka
messing around with structured org files (and captures, with doct)
writing a passwordstore-like script for with an age backend
I’m tinkering with “window tags” as a workflow concept. With a recently added “dismiss” motion it feels fairly natural now, but some dynamic flow is still needed to be feel just right™. Also, I tricked i3blocks into being a lemonbar formatter. Here’s a gif of some tag stuff and a screenshot of new panel fun: tags gif (10mb 2K) – panel shot
I’ve been working on enabling direct reads and writes from/to our S3 blockstore in the browser for Peergos. The Java SDK from Amazon doesn’t expose this so I’ve had to roll my own request signing class, which essentially amounts to my own S3 SDK, but in a single class, not 60mb of jars. This should make hosted Peergos much more scalable and faster. The cool thing is for writes we can force S3 to verify the sha256 of uploads. This allows us to maintain the blockstore as content addressed.
Setting up a nomad cluster to run VMs as provided from various vendors, but run the VMs with qemu so I don’t have to hate myself running virtual box.
At work, writing a server daemon that uses only message passing to store state.
At play, writing a first draft of dxpb (my project to improve Void Linux’s build situation) using Clojure and building on top of as many existing components as I can, trying to see how fast I can do it.
For work, I’m getting an opportunity to do a deep dive into specific parts of a cloud platform I don’t have much experience with. This is triggering some of the fun I get from learning a new Fromsoft game - the whole “Okay, what have they renamed everything” experience, but at the same time noticing the things they’ve refined on and objectively improved from the platform I already know.
For personal stuff, I’ve got my Pytorch StyleGAN implementation working - or at least well enough I’m convinced I did it right. I can’t justify taking my GPUs off COVID folding for long enough to do the full number of passes over the dataset, but hopefully by sometime this weekend I’ll be able to get the code cleaned up well enough to push the sequel to ProGAN for Humans and finally get started on StyleGAN2 next week.
At start of this month I’d kept goal of running 100+ miles this month. So here we are in last week of April and I’m shy of 6 miles to achieve my target. Very happy and proud of myself if you ask me. And to make it even more memorable, I’d been running all these miles within home in our hall (25 ft in length). I’ve set even more fun challenge run over the weekend to celebrate this milestone. More on that later!
Work
Finally I’m able to get Jira Data Center Single Node up and running in our TEST environment. Now working on SSO Integration with Jira. Need to take care of few issues for this to work.
Mainly bug fixes and thinking about a way to distribute a side project I am working on: atbswp, it’s a minimalist macro recorder, you can find more details here
Hopefully swapping out some of the weatherproof boxes with power sockets inside, for more sealed ones.. due to the repeated habitation of said boxes by sneks.
I’m working on adapting to my new Emacs workflow as I’m currently giving EXWM a shot.
I must say, I am really loving this so far. It’s so awesome to use X windows as normal Emacs buffers.
Besides that, I’m testing out random Rust programs I’m coming across. I love trying out different programs, especially when holding off from doing my own programming. (When you do something for fun, and are naturally a very lazy person, you don’t exactly have the best work ethic.)
Got a day off tomorrow so I’ve been hacking away on my FPGA video card a bit more. Currently hesitating to try out the OSS toolkits some people have mentioned the other day (OTOH I spent a bunch of time yesterday on an unrelated project trying to get Typescript working and ended up not making “real” progress…)
really feel like I’ve been toiling away on tooling rather than real progress on projects recently. Not a great feeling
Finding a jerb. Who wants to shove money down my pants in exchange for code?
I’ve been becalmed for several weeks on my language that skips Assembly and C and translates directly to machine code (paper [pdf; 25 pages]; repo; blog post).
The problem: I’ve been using unsafe addresses all over the place even though I always planned to switch them to fat pointers which protect against accidental memory corruption. It was just easier to put off that concern while I got things off the ground. But now I realize that fat pointers don’t fit in registers (which my language exposes) and so can’t be returned from functions (because functions can only return registers). So I’m having to go through and rewrite a bunch of function signatures and call-sites, while moving local variables all over the place from registers to the stack. All while programming in machine code and having to adjust struct.field offsets by hand. It’s stressful work, because I need to fix things everywhere in a program before I can run any of its tests. I know that if I’m not careful up front I’m liable to end up in a hole I can’t debug myself out of.
The upshot: if you look at my repo there isn’t a lot of recent commits. But that’s because the work has been happening on a private branch. Most of the repo has been translated, with the exception of the compiler from the HLL to machine code, the biggest program written in Mu by far. There I’m about 40% of the way through the translation, but none of the 40% has been actually verified, so who knows.
Being forced to stay home isn’t helping. My presentation for the conference next week (free to join!) is also taking away some momentum. But that’s actually going well.
I’m going to just keep at it. It’ll take as long as it takes. And hopefully I won’t need to do stuff like this again for several years at least.
Hi akkartik, I recall interacting with you once on HN about one of your languages. I’m struck by how off the beaten path the kind of work that is. If you don’t mind me asking, do you have a long-term vision/goal with your language projects, or is it something you are just doing out of interest? I can sympathize either way.
Hope you are well.
edit: I see your blog gets into this. Very cool work. We need more radical experiments like this.
Why thank you! Until about 2015 or so I would periodically lose interest in my side project and cast about until I found another. Since then, though, I’ve been on a single track, so I feel very fortunate.
I found our old conversation in my records. Good to see you here, and I hope you are well too. Feel free to email or DM me if you have more questions.
If you like that, you’ll probably like our collection of the work going on in that space.
When into bootstrapping, my first goal was to get above tedious manipulations. Learning from Lisps and meta languages, the idea was to automatically bake in some ability to do metas that let me write higher-level functions that converted to lower-level. Maybe automate things like offsets, too. If too primitive and close to machine, then the developer has to essentially think like a machine on a regular basis for improvements. It just seemed like we as human beings shouldn’t be doing that. Some people like it, though.
I did think I’d want to see, REPL-style, what each high-level statement generated. That way I could keep stuff efficient and/or secure. Also good to be able to just inline some hand-written ASM or machine code if I’d want to do it better.
My personal goal was the minimum number of steps I could do to immediately get away from ASM. The design called for an interpreter hand-written in minimal ASM or machine language supporting basic expressions, structs, function calls, arrays (for file I/O), basic I/O ops, linked lists, trees, some operators on them, and substitution-based metaprogramming. All aren’t strictly necessary but are convenient. With that in place, the rest can be layered on top with the interpreter executing or, if output is logged, used as a pseudo-compiler. The simplified tools bootstrappers collected, such as for ELF’s, could be ported to that foundation to run similarly. Dropped the second we get TCC, LCC, or whatever through it. I’d have automated the TCC port using existing tooling for C transforms followed by distributed eyeballing and equivalence testing.
That’s where I left it. I enjoy seeing the solution more than tedious work of building it. Impressive that you’re still keeping yours low-level despite the challenges involved.
“ because I need to fix things everywhere in a program before I can run any of its tests. “
Quick note to anyone in this space. Another advantage of an interpreter with meta-programming is that you can build and test each feature in isolation. Then, you can test them together. The meta can support that with some code reuse by having two modes, running and testing. Kind of like ifdef’s in C codebases but integrated with language. Even bootstrappers should be able to enjoy productivity boosts.
Although, it really depends on one’s goal and associated constraints. Mine was just defeating Karger-Thompson attack on C toolchains in widely-verifiable way. Some like akkartik want deep understanding and control of low-level details from bottom to top. I still think they could use extra features if only to be like having IDE support saving time on tedious stuff.
Yeah, Mu could certainly use extra features over machine code. We already have struct offsets. Try this out on a Linux machine:
Screenshot with syntax highlighting.
It’s a trivial program. It writes to a struct field, then reads back from it and returns the value.
In your language, can functions return results in more than one register? Is there a way to express that a single two-word value should be stored in two registers?
Great questions, thanks.
Yes. Functions can return results in as many registers as they want. The registers of the results are part of the signature just like the types.
The calling convention is for the callee to preserve all registers except those that return results.
No, there’s no way to manually perform scalar replacement at the moment. I don’t yet know how to preserve type-safety in that situation. It would likely require some additional syntax, and it’s not obvious that the benefits outweigh the additional complexity.
I know the feeling. My regex compiler has the same issue. If I pass invoke the wrong methods in asm, I often just get a null pointer in the library. The next step is a bytecode dump that can’t be parsed by the standard tools. And in a compiler (or at least a compiler written by me…), you can have to change far too many entangled things before you get a useful result.
The upshot is that I’ve ended up twenty commits deep in a branch, slowly working to just get something that produces runnable code. Eventually I get there, and I can start to worry about the mundane details, like “do my tests pass?”
Reading and re-reading more books from The Domino Project and applying the lessons therein:
It’s a year of revisiting proven works instead of constantly seeking out new ones.
I also want to prototype my first ActivityPub app: a PWA (progressive web app) focused exclusively on posting authoring. Something to help me focus on creative work without getting distracted by new posts in my feed.
How do you get started with The Domino Project? It seems like a cool project, but it seems like all the information is spread out in different “blog posts”?
I wish I could recall but it was years ago. If I had to guess, some combination of blog posts and podcasts bringing the individual books to my attention.
Alright, thanks ✌️
I wrote a toy webserver that listens on a port and responds to only one request before moving to a new port. It also tells you where it’s going to migrate to :D
So far it’s deployed as a capture the flag type exercise, but I might wrap some service discovery systems around it in order to learn more about that domain and the tools available.
If it was a gopher client you could call it whack-a-mole.
Interesting! I wonder how seamless you could make the experience be using JavaScript. Another idea I just had would be to make it into a choose-your-own-adventure game: you’d make a choice by going to a different port. I’m not sure if that would be any more fun than just clicking a link, but it could be a fun demo.
Yep, a client that implements that seamless-ness is required for finding the flag in the current version - in addition to the next port it will listen on, the server also provides a single, random, indexed value from the flag. In order to solve the challenge you would need to write some code that follows the breadcrumb trail for long enough to suss out the flag.
Glad you found it interesting, it was a lot of fun to write. Python’s aiohttp is super neat.
This is really cool :)
Hopefully finalizing the write-heavy, heavily-shared object store for the next generation of our product. It’s complicated because it’s not just a flat database; it’s closer to a strongly typed graph database but for reasons it is built on top of a classical relational model in PostgreSQL.
Every graph-oriented database I experimented with didn’t have one or more of the following required features:
If any crustaceans know of any graph-oriented DB that meets these sort of requirements, I’d love to hear about it, thank you.
Over the weekend I experimented with automatic code generation. We have a large, but pre-defined, set of object classes to be stored, which is amenable to code generation but…the predefined list isn’t expected to change all that much, so it boils down to the classic problem of spending a lot of time to build the tool and save time later or not.
On the personal front, I started a new book series with each of my kids. On nice days, we sit outside and I read a chapter to each of them out of each of the book series we’re doing (we also do a chapter every night at bedtime). The older boy and I just started on Percy Jackson and the younger boy and I are doing The Last Firehawk series. We’re making good progress through those books.
(If you have a preschooler who really wants to like Harry Potter like their older sibling(s) but doesn’t have the staying power to make it through long chapters and complex plots, The Last Firehawk might be for you.)
There’s a big community for randomizers.
You reverse engineer an old game, put the items in different spots, and people organize races on the resulting game, having to route on the fly where to go next depending on what they know they can do (in-logic), what they know they can do with glitches (out of logic), and what they can’t do (item locked).
(I should write a full-length piece on this, fascinating stuff and display of technical skills.)
As a speedrun and rando enthusiast, I follow a few runners, specifically The Legend of Zelda: Ocarina of Time Randomizer runners. One of them (Marco) said that if there was a ladder (ELO-style ranks for 1v1 races) for OoT he’d do more rando.
I’m out of a job, I can’t get out of my apartment, so I went and did it: Kaepora is born. A race planner with Glicko-2 leaderboards and a Discord bot.
The first public race finished as I wrote this, it’s a success. 16 people were competiting this evening just for the sake of running OoTR and knowing who’s the fastest. Absurd seeds were had, good and bad routing decisions were made, some were lucky, and some had their first bomb bag on 50 skulls.
This is already peak niche nerd stuff, but next I’ll open the randomized settings randomizer league. Twice the rando, twice the fun.
Yes you should. It’s fascinating stuff, and the LTTP/Super Metroid crossover randomizer blew me away.
I have an extense to-do list that increases its content every day, so my weekend is going to be to try and make a lot of tasks, give my server a bit of maintenance, and more time for my projects.
Also, writing a ton of drafts for my blog so I can keep publishing stuff.
I have been spending some time of my lockdown time writing a multiplayer text game for IRC: https://pink-dragon.surge.sh/
It is now running on https://tilde.chat/
Looks pretty cool. Out of curiosity what’s tilde chat
It is an IRC network for a dynamic community, sharing some affinities with the much older sdf.org
If you like IRC, you’re welcome!
https://tilde.chat/ https://sdf.org/
In my free time, I’m trying to write daily for 100 days as a part of #100DaysToOffload. As well as reading through some Greek myths by Stephen Fry.
Thanks for linking 100 days to offload, that’s pretty cool
I’ve got to try to figure out how to focus, mainly for work, but also for some personal stuff. Between isolation and being three months into a new job, I just can’t sit and focus on my work. Not “oh man after three hours of meetings in a day, I’m only coding five more hours”; more like “I’m spending six hours in front of the computer and not even being productive for 30 minutes total”.
At least my therapist has something to work on.
Definitely felt this – GL solving it on your end.
Thanks. It’s pretty hard. I’m gonna take a day or two off, but I’m not exactly sure what to do on that day off.
I’m working on building an electronic chess board, apparently it’s not a new thing but it’s a super excited project to work on. I documented the process on my personal blog.
Pretty neat! Keep up the good work.
thank you!!
@home: Prompted by @hwayne’s request for “FactoryBot” style APIs for property based testing, I’ve tried my hand at it. Over the weekend, I created a pretty small POC. I actually ended up spending more time on stereotyped tests than generators, but I have ideas for both sides of it.
Short version, the idea is that you can write things like:
I have a bunch of ideas for improving the syntax for declaring variables, and there’s also a lot of properties that you might want to check for functions. Beyond that, anyone who looks at the code will realize I don’t know much about Choco Solver, which I’m using under the hood, and I need to learn more about that.
Continuing to make progress on my personal site. Deployed it with Phoenix Liveview yesterday, really no reason for the Liveview other than the fact that it’s fun :)
I’m also doing chapter 2 and 3 of the https://grox.io Elixir course.
(8 weeks ago… a lot longer than I remembered https://lobste.rs/s/epx9km/what_are_you_working_on_this_week#c_bnj6ad)
$WORK: It’s paper deadline time again! We’re submitting one paper on Thursday, so this week is all about wrapping that up. Then the next day, I have a final project presentation for a class before doing the last sprint on another paper due early May. After that, I start my virtual summer internship.
Quarantine has been getting to me more lately, so I’ve been resolving to try to have more of a schedule and get more time outside in the sun. I’m finding it pretty hard to stay motivated sometimes, but having this work schedule with the papers helps a lot, surprisingly.
$HOME: I had a lovely anniversary weekend with my wife, and I really enjoyed having time completely away from work, so I think I’ll keep that up. That said, I’ve been reading a professor’s blog and am feeling very motivated to do more writing on my blog and I’d love to start that up again. I felt very accomplished after finishing the last post I wrote up.
Other than that, it’s just going to be more crime drama bingeathons and relaxing.
This week I want to start an art project. I either want to do an illustration project or a music project with my guitar. Maybe even both. I’m not sure what I want to do yet though. Anyone have any recommendations for someone who’s learning illustration and/or guitar?
$HOME: Finally got my laptop back from HP, who repaired my laptop hinges for free. I backed up/wiped nearly everything before I sent it in, so I set it up to now dual-boot Windows 10 and Linux. I’m wrapping up lessons learned from “No code after work”, so I’m ramping work back up on my 2D C99 game engine.
$WORK: I’ve been struggling for quite a few months, but putting in the effort so things are slowly turning around.
I’m writing a small typing trainer in Go to get used to my NIU 40% keyboard
https://github.com/noqqe/typist
Working on some iOS client work. I’ve decided I should really become fluent in Swift so I’ve been working my way through the swift programming language book, and doing some advent code along side. I kind of want to start building an NES emulator! Seems like fun.
I can’t help but wonder when I’ll just start putting more time into Rust. But for now, I think the payoff is something I need to use as a guide for my decision making.
Also, I just signed up for a write.as pro trial. So I’m going to try and set up my blog, and start writing!
I’ve been timetabling my days to give me some structure so that I don’t waste the longest holiday of my life (I’m meant to be on exam leave, except exams have been cancelled). My day looks something like this:
Those activities usually take me up to about 4, after which I’ll go to the allotment for a bit, and then come home and work through K&R. I think having some kind of a structure is making the whole thing more enjoyable.
My advice: choose one of those things and do it.
I once had a summer (in high school - which goes to 20 year old where I’m from btw) where I decided not to work and just tackle my list of things that I wanted to do.
By the end of the summer I had a firefox session with a few thousand tabs and anxiety about all of the things that I needed to know but didn’t yet. The culprit was that instead of treating the list as a stack or a FIFO queue I tried to make a priority queue, calculating the priority wasn’t decidable and the net effect was just to add more things to the list. I have never had a busier summer, it felt like time was speeding up exponentially.
If I had been able to focus I would maybe have learned a single useful thing instead of an encyclopedia’s worth of trivia that I was unable to manifest (i.e. pointers to more information rather than actual skills).
I think that the only thing that’s actually going to be an issue for is the German, as most of the other things aren’t so much about learning as doing. In previous summers I’ve tried sticking with a single thing for the whole time, but I can’t concentrate like that.
In some ways, I am treating it like a FIFO queue, just with multiple queues running simultaneously. Essentially I want the structure to still be there throughout the day, as having nothing compulsory to fill the day with and this long holiday, before going to study a course without a huge amount of contact time seems like a recipe for dropping out. As some of the activities near completion (“lessons” one and two are nearing the end even now) I can choose another thing from my backlog or increase the amount of time on one of the other tasks.
Thank you for the helpful response and for the concern though. I think that for now I’ll stick with my current system, and if in a couple of weeks I feel like I’m achieving nothing I’ll double down on one and you can say I told you so :)
Probably you have much better habits than me to begin with. Good luck! :)
How’s that coming along? Is this your first foray into C?
As good as. I’ve modified the code for various programs before, but that was more guesswork when something didn’t work quite how I wanted, and I’ve followed a few tutorials for C projects, but never fully understood what was going on when the less intuitive features were whipped out. I figured it’d be a much smoother experience and allow me to work on some other things I wanted to work on if I actually understood the language. I’m having quite a lot of fun with it!
Continue with a multi-month effort to move a big rails production application from Heroku into EKS.
Reading some more pages in the analects. Working, looks like a simple sprint. Studying topology (applying for university and was missing topology from my transcipt).
I’m currently funemployed (by choice) and am treating the time until I seriously start looking for a job as ongoing programmer vacation, kinda.
I’ve been working on enabling direct reads and writes from/to our S3 blockstore in the browser for Peergos. The Java SDK from Amazon doesn’t expose this so I’ve had to roll my own request signing class, which essentially amounts to my own S3 SDK, but in a single class, not 60mb of jars. This should make hosted Peergos much more scalable and faster. The cool thing is for writes we can force S3 to verify the sha256 of uploads. This allows us to maintain the blockstore as content addressed.
Building a javac plugin that warns when
nullis assigned to@notnulltypes.Setting up a nomad cluster to run VMs as provided from various vendors, but run the VMs with qemu so I don’t have to hate myself running virtual box.
At work, writing a server daemon that uses only message passing to store state.
At play, writing a first draft of dxpb (my project to improve Void Linux’s build situation) using Clojure and building on top of as many existing components as I can, trying to see how fast I can do it.
For work, I’m getting an opportunity to do a deep dive into specific parts of a cloud platform I don’t have much experience with. This is triggering some of the fun I get from learning a new Fromsoft game - the whole “Okay, what have they renamed everything” experience, but at the same time noticing the things they’ve refined on and objectively improved from the platform I already know.
For personal stuff, I’ve got my Pytorch StyleGAN implementation working - or at least well enough I’m convinced I did it right. I can’t justify taking my GPUs off COVID folding for long enough to do the full number of passes over the dataset, but hopefully by sometime this weekend I’ll be able to get the code cleaned up well enough to push the sequel to ProGAN for Humans and finally get started on StyleGAN2 next week.
Setting up a proper home office instead of just using my gaming rig, which is long overdue.
Trying to keep making useful stuff at $WORK, occasionally instead of the stuff that people think that I should be doing.
Playing too much XCOM Chimera Squad.
Personal
Work
Mainly bug fixes and thinking about a way to distribute a side project I am working on: atbswp, it’s a minimalist macro recorder, you can find more details here
For
$CLIENT1:For
$COMPANY:For
$HOME:Oh, and how could I forget, for
$HOME:Hopefully swapping out some of the weatherproof boxes with power sockets inside, for more sealed ones.. due to the repeated habitation of said boxes by sneks.
I’m working on adapting to my new Emacs workflow as I’m currently giving EXWM a shot.
I must say, I am really loving this so far. It’s so awesome to use X windows as normal Emacs buffers.
Besides that, I’m testing out random Rust programs I’m coming across. I love trying out different programs, especially when holding off from doing my own programming. (When you do something for fun, and are naturally a very lazy person, you don’t exactly have the best work ethic.)
Finish a book proposal for a book:
Elixir in Practice - Build an e-commerce platform with CQRS/ES and Elixirand trying to find a new job.
Attempting to learn about WebRTC and searching for a way of creating a bespoke interface for members of teams to be able to partake in remote events.
Got a day off tomorrow so I’ve been hacking away on my FPGA video card a bit more. Currently hesitating to try out the OSS toolkits some people have mentioned the other day (OTOH I spent a bunch of time yesterday on an unrelated project trying to get Typescript working and ended up not making “real” progress…)
really feel like I’ve been toiling away on tooling rather than real progress on projects recently. Not a great feeling