This is the weekly thread to discuss what you have done recently and are working on this week.
Be descriptive, and don’t hesitate to ask for help!
Have a week to myself and finally figured out how to open the (first) novel I want to write. (I ended my blog with an excerpt.) About 40,000 words toward a target of 120-140k. (I expect the series to come in around 600k-800k.) I’ve been designing the characters and world for years but it wasn’t until last month that I knew how (or I should say, when, because it’s a more compelling story if I start in the middle when the main character is in her early 20s) to start the damn thing.
Finishing up reading the infamous Dragon Book. It gets a lot more interesting once you get through the parsing purgatory in the first third. Compilers are fun.
Looking at job opportunities (recently finished a contract). Trying to decide whether I want to move back to the East Coast. Chicago’s great, but there aren’t as many opportunities out here.
My main aim this week is to keep hitting my daily exercise goal, which mostly involves badgering my mate to go to the gym (we guilt each other into turning up, until it becomes a habit again) or going for walks and catching up on podcasts.
Also away from backlight rectangles of light, I’m also aiming to get the dinghy in the water and get the project car back on the road. Car should’ve been fixed at the weekend but like an idiot I replaced the camshaft sensor when it needs a new crankshaft sensor. (Duh!)
Successfully segregated my home network onto VLANs last week, now I just need to firewall the “management” VLAN from the rest to make use of that segregation.
Successfully segregated my home network onto VLANs last week
Nice. How many VLANs have you settled on and have you done anything specific to handle “IoT” type devices?
I’m using VLANs at home for my DMZ and guest internet access but haven’t quite worked out how best to handle things like my set-top box, “smart” TV, Sonos speakers, etc. Ideally I’d like to keep all of those on their own VLAN but that stops things like being able to control the Sonos from a machine on the trusted VLAN (it’s a nightmare getting Sonos devices working across VLANs). Chromecast and Apple TV have similar problems (although those are a little easier to workaround with some Avahi magic). Argh.
Currently I’m up to four VLANs:
The only thing currently firewalled is the Isolated VLAN however, the others can happily reach each other atm. (First step was getting everything on the correct VLAN with the correct DHCP pool assigned.) IoT things have so far been dropped on the Isolated network as they only require internet access, as well as things like my Xbox 360 that needs internet but nothing else internal.
Choosing whether I want to build multi-DC stateful systems full-time, write a book on distributed systems or rust, and/or just travel around and work on well-specified rusty high-performance persistent structures and distributed stores. Good times ahead in any case :) But before I have specifics, it’s a bit stressful for me to deal with apartment relocation stuff out of NYC without knowing where I’ll be in a month, how hard I should be looking for an apartment in Berlin, etc… I don’t want to be fully mobile with a full-time gig. Head is spinning, mostly in good ways.
If you don’t mind, what was your journey like for learning about formal specification/verification (TLA+) to creating projects using it, etc.?
I’ve turned to formal methods in anger after dealing with too much stuff that doesn’t work, is hard to debug, and is unclear if it even CAN work. The breaking point for leaning into TLA+ was pretty recent, I’ve heard about it for years from friends in the distributed systems community, but have never really felt like it was worth the effort for what I’m building. The breaking point came a few weeks ago when I was writing a lock-free log-structured storage mechanism for rsdb. I needed to implement a number of structures from a paper, where they were not very thoroughly specified. I got the thing to pass some simple smoke tests. Then I ran a script that juggled thread niceness and it blew up. I fixed a bunch of bugs, and I’ve fuzzed tested it, but there is still zero confidence that the thing is correct. At the same time, I saw hwayne’s TLA+ guide that he posted here on lobste.rs, and finally decided to dive in. TLA+ helps me to know that I’m at least pursuing something that is possible. I also have a consensus algorithm that addresses a livelock issue I don’t like about raft, but it would be downright crazy to rely on this without proving that it is correct. I’m still a beginner, but it’s helping me to feel better about how I’m spending my time.
Brilliant, thank you for the thorough answer. I’m starting to go through something slightly similar (‘too much stuff that doesn’t work, is hard to debug, and is unclear if it even CAN work’). After having proved an algorithm for example, how do you ensure that the implementation matches?
I’m still a beginner, but it’s helping me to feel better about how I’m spending my time.
I’m looking forward to this. I feel like there’s too much guessing/hacking around in software engineering, DevOps, etc. sometimes.
After having proved an algorithm for example, how do you ensure that the implementation matches?
This is one of the things I’m most interested in right now. I’m totally dissatisfied that people building stateful things are resorting to TLA+, which isn’t real code, or black-box probabilistic testing like jepsen, which has very low bug:compute time ratios.
The middle ground is simulation. I’m kind of obsessed right now with the idea of “simulatable systems” that are also amenable to exhaustive analysis while writing them in non-specialized programming languages.
My theoretical knowledge is totally lacking in this space, but I’d basically like to specify things as state machines with certain rigid transitions based on messages that may arrive arbitrarily late. I am very much in the researching what research to research phase of this haha, and I’m almost certainly going to hit some impossibility results or something that will change my goals for what I should aim for, but right now I’m naively plowing ahead :)
But I know it won’t all be in vain, I’ve built some probabilistic simulators that vastly outperform black-box partition testing by controlling clocks and delivery times/orders from a simulated transport, but I would like to push this farther into the direction of exhaustive testing.
Quite a few of the things you’ve said align with what I’ve been thinking about, and I completely understand the ‘researching what research to research phase’ haha I’ll keep tabs on what you’re doing (from your GitHub) account and in the odd chance I come across something I think might be useful, I’ll shoot it your way.
Do you have any links to the probabilistic simulators that you’ve made so far? Also I couldn’t find the TLA+ files in either of the repos you’ve got up.
The last few weeks I’ve been working to expand our on-call rota. To avoid dropping people in on the deep end, I want them to to have some training. Since we don’t have many incidents, we have to create ones so I’ve been setting up a weekly training session inspired by Failure Friday at PagerDuty. To avoid it being too much of a timesink we try to limit it to 1 hour, and we only break one thing at a time. Once people are getting more practice (and we start running out ouf scenarios!) we might break multiple things at the same time. Further, so far we have focused on incidents introduced by operator error or third-party services having a bad day. We’ve not played with active saboteurs yet, which could be interesting.
Supporting this I’ve been focusing a lot on documenting checklist for the process we want to use when responding to incidents, which resources are available to resolvers, which are critical services, and incident severity levels. For some sections I’m pointing to https://response.pagerduty.com/ but pointing out where we differ due to size.
Finally got back to Inbox Zero after being at 900+ for a few months. The feed reader’s a wreck, but at least now I’m not missing messages from friends and family in the noise.
I’m starting a Chicago meetup for saas/online service businesses, probably named something like “Chicago Online Services” to broadly catch subscription-based businesses. There’s a decent number companies doing things Chicago-style and I’d like to encourage a healthy ecosystem and community.
And I’m doing motorcycle maintenance to take it out as soon as the weather stabilizes a bit (Chicago spring is less of a transition and more of an oscillation between winter and summer). The tachometer stopped working in the last week before I put the bike away - probably I just need to lubricate the cable (rest of the panel + electrical is fine), but there’s a chance I’ll have to replace the cable or fix/replace the instrument. And a bit of yak shaving, I need to measure some bolts so I can get the right-sized wrench so I can change the oil. Any other riders with spring-prep checklists to share?
Chicago spring is less of a transition and more of an oscillation between winter and summer)
Often true, and apropos. In March, we had a literal 80s-to-30s day with serious rain at the end of it. Went out in shorts, had to get a ride home.
[Comment removed by author]
Was it from that Incremental Construction paper?
Well, that’s even better. Much more rewarding experience. :)
On Friday I impulse bought like 10 square feet or so of plain wafers? Not vanilla wafers or wafer cookies, literally just ten square feet of the insides of a KitKat bar. Over the weekend I ran a couple experiments on how they handle moisture migration and they hold up pretty well, so this week I’m going to be turning them into chocolates. First batch will be lavender truffles, second will probably be ‘KitKats’ made from ganache.
Nothing too exciting on the computron side, though. Not all weeks can be computron weeks.
At $WORK, webdev, ugh. But somebody’s got to do it, and my specialty (complaining about technical debt) is in low demand right now because we’re working on a new, greenfield project.
At $HOME, continuing to plug away at my toy game engine. This weekend I got clipping against the heightmapped ground mostly working; the next two features in line (not sure of the order yet) are heightmap LoD and per-vertex material properties (so the whole thing doesn’t have to be horrible programmer-art barber shop tile).
Planning on submitting some tiny PRs to a golang lib we use all over the place at work.
I finally launched a tiny macOS app (open source) yesterday so it was a good way to start the week.
I’ve been learning about WebRender and started to make some pull requests. I’ve done a couple of documentation improvements and got a minor API change accepted that makes an important part of the API easier to understand.
Learning WebRender has been a good bit of fun and it seems pretty promising. (I’m not interested in using it as part of Servo…)
My daughter (6yo) has a 2.5 week break between terms at school, so the next couple of weeks will be interesting.
Have a good week!
Digging out some code I wrote years ago that auto-skins simple videogames, because something along those lines turns out to maybe be useful in my new-ish job.
Con: it seems that J2ME apps on featurephones are no longer a contemporary development platform, and even getting that stuff running in a simulator has bitrotted to hell. Pro: it seems possible to excise the core constraint-based skinning logic from the J2ME stuff, in part because this was written early-ish in my PhD when I was apparently quite conscientious about commenting and structuring and such. The code from the last years of my PhD is a mess, but this code comes with installation instructions and a detailed README. Amazing.
Last few weeks have been wishy-washy waiting for contract stuff to be sorted out, but finally have a contract extension sorted and payments coming through again, so I’m back to working on what I’ve been working on for the last 5-6 years; building disease surveillance, monitoring and alerts systems in the humanitarian space. Trying to get a first pass at the desktop application going (built on QT using a modified version of the web UI in QWebEngine) and getting syncing and data submissions simplified and compressed enough for the desktop app to be usable in places with extremely bad internet or where they’re using tethering.
Other than that, starting the paperwork to get citizenship documents ready for Canada for my kids in case my application for ILR here the UK doesn’t get approved when I apply at the end of this year!
Doing a bit of digging into CMake/CPack for packaging of the web application to make installing easier.
And other than that, chipping away at a 100 item long backlog of issues in our mobile and web apps.
So yeah, uhm, lots, probably too much.
Doctor just gave me two more months on the csst, so no more skiing for me, but that does mean I get a few extra paychecks from the ski school..
At work I’m balancing bug fixes with adding new features and slowly rewritting it to a mix of mvc and functional programming (Full oop is too much for me to handle, but it isnt to hard to rewrite some of the procedural code in a funtional way, which is much nicer anyway)
Dealing with Second Batch, born on Wednesday. So far, so good, although all I have the brain capacity to deal with these days is lousy movies in the middle of the night. Maybe this week I’ll spend some more time with my toy project (an MPEG-4/ISO parser in OCaml).
I’m building a react native style framework in go. Currently working on the constraint layout system.
Containers on Google; Kubernetes.