This week I’m working pretty heavily on 44CON, on the call for papers, our house rules, t-shirt designs and our 44COIN project.

The CFP is going fine (but we need more speakers, so if anyone here has anything interesting they’d like to talk about I’d love to see it).

We’re working on something we’re calling house rules, which is an attempt to build something that helps set guides for behaviour, escalation etc. without falling into flaws/traps commonly associated with codes of conduct. This week the draft house rules are going for final draft review internally, then being pushed out to various groups for external review. Should have them ready to publish in April.

Last week I did a ton of sticker designs and the first prints are coming back this week. I’m doing T-Shirt designs this week, which are a mix of 1984 and DDR themed. Last year our crew were “alternative domain admins”, this year they’re going to be officials of the “Ministry of Freedom” as a nod to Orwell’s classic.

Finally, I’m testing our 44COIN, which is a little hardware project for people to build which is also part of the ICO we’re going to run at the event. People who’ve been to 44CON before will know that this will be run entirely sarcastically and as a parody rather than a serious investment opportunity.

Last week was a sprint to finish stats assignments for end of term at the London School of Hygiene & Tropical Medicine:

1. For Analysis of Hierarchical and Other Dependent Data, an analysis of how patients’ CD4 count trajectories varied depending on drug and three variables collected at baseline, using a mixed effects model and generalized estimating equations,
2. For Bayesian Statistics, a ficticious analysis combining data from a randomized controlled trial, a case-control study, and a cross-sectional study – it’s a bit hard to believe, but even though they study completely different probabilities, the odds ratio of outcome versus exposure (or equivalent ratio of exposure versus outcome!) is a common parameter.[1]
3. For Survival Analysis, Kaplan-Meier and Cox regression analysis for differences in time to death for patients diagnosed with Follicular Cell Lymphoma, by age, stage at diagnosis, and hemoglobin level (g/l) in a ~30 year cohort study, plus analysis of competing risks of relapse or death not from relapse.

This week is:

1. figuring out if the data’s going to be ready for my summer project, and lining up contingencies if not,
2. job marketing & networking, both in public health and back into the London software scene,
3. as the English call studying, “revision” of the past two terms of coursework. Comprehensive exams are in early June.

[1] If you’re really interested, the probabilities in question are:

1. For RCT, we administer an exposure, count outcomes, and then estimate probability of an outcome, given an exposure – Pr(outcome | exposure). So in a non-inferiority trial, we’d measure ,Pr(Death|Drug A) vs. Pr(Death|Drug B).
2. For case-control, we collect some number of each outcome (called “cases” and “controls”), count exposures within each group (cases or controls), and then estimate probability of an exposure, given an outcome. So, again in this example, Pr(Death|Drug A) vs. Pr(Death|Drug B). (Usually, case control is about some non-administered exposure…but this was an exercise.)
3. For cross-sectional, we collect some total number of people, count how many had every combination of factors, and then estimate the joint probability Pr(Outcome AND Exposure) or Pr(Outcome)|Pr(Exposure). So, again here, Pr(Death)Pr(Drug A), etc.

Spent the weekend playing with hardware devices. Reached a point where I can control a ceiling light in the way I want to, with a Sonoff Basic. (Only destroyed one of them along the way too, score.) It’s currently soak testing on my dining table with a table light, but the plan this week is to install it into the ceiling of my office. (Basic gist, relay in the Sonoff controls the light on/off, and the light switch on the wall completes a circuit on GPIO14 pin to toggle the relay. Plus wifi controlled from Home Assistant/HomeKit.)

Also reverse engineered a battery receiver for my wireless doorbell (Wilko’s cheapest), and wired that into a Raspberry Pi successfully. When someone rings my doorbell, GPIO pin is pulled high and go-doorbell daemon emits the event into MQTT. I then have a script in Home Assistant that pauses my Sonos speakers, plays a doorbell sound through them and restores whatever the speakers were doing beforehand. Obligatory Tweet about the project

This week I’m aiming to have control of my office via HomeKit/Siri. (“Hey Siri, turn the office on/off”.) That just involves a ceiling light coming on, and a wall socket turning on. (via Sonoff Basic & Sonoff S20 running Tasmoto firmware.)

I’ve also promised someone a blog post about how I hooked up the doorbell, so I’m aiming write that up at some point too.

And waiting to see what cheap electrical goodies from China arrive this week. Downside of cheap, month long shipping: I forget what’s coming; Upside: surprise in every delivery!

This is my first time posting here.

I am looking into hardening certain aspects of Wallaroo’s Python API and improving the error messages when improper pipelines are set up. Hopefully this will round out many of the sharp edges I hit when getting up to speed with Wallaroo’s python support.

After that I’m working on some benchmarks to identify bottlenecks in horizontal scaling of clusters. This work is a bit open ended but I expect to be doing a lot of profiling and system call tracing once we’ve narrowed things down. There are a lot of knobs to test and turn here but we’ll start by creating a simple baseline with cheap computations and then find where our current sweet spot is when sending work across the cluster (for extremely cheap computations, I/O overhead dominates). I’ll look forward to reporting back next week.

I’ve been staying busy in Rust land.

I recently took over maintenance of the z3-sys and z3 crates from Graydon and have been working on them. I’ve published a new version of z3-sysand I’m working on updating the z3 crate. (Repo.)

I’ve published a long overdue update to my lldb-sys crate that brings is close to being up to date, and this week, I’ll update the lldb crate for the newly bound features. (Repos: lldb, lldb-sys.)

I’m slowly working on some RDF-related stuff as well as Linked Data, but not sure what I’ll get done this week on that yet.

The final thing for this week is that I’m writing something to use some code borrowed from serde-json to provide serialization / deserialization between Rust and JavaScriptCore. I maintain bindings for javascriptcore-sys and javascriptcore and would find it useful to use serde to translate values back and forth.

Planning out some long-term work at Wallaroo Labs, and writing a blog post about how our Python API for writing stream processing applications works under the hood.

I ported my compressor to use AVX2. Most of it was find and replace because most SSE instructions have a same-but-twice-as-wide partner in AVX. Not all of it was that easy though, SSE has some instructions that work on the entire 128bit register (e.g. PSHUFB) while their AVX counterparts work on 2x128bits rather than 1x256bits. Took some staring to figure out which parts were actually broken and the fixes are ugly and slow but it works now.

The AVX implementation is somewhat faster than the SSE implementation, which I’m pretty sure makes it the fastest decoder in the world. Compression is not awesome but I have some experiments to try which may also end up making it faster as a bonus.

I’ve been having a lot of fun working on this and I’ve been thinking about how to move my career towards this kind of work too. I like algorithms and low level performance tweaking with more science than engineering. It seems to me that jobs like this simply do not exist (obviously they do but they’re extremely rare) so I’m investigating starting my own company to work on it. Going to try to schedule some phone calls with people that have taken similar paths and other people who might be interested in buying it and see what happens from there.

My little timeseries DB called henhouse from a previous project has finally been open sourced. Maybe I’ll investigate using seastar to make it even faster.

I’ve been working on a new core data structure for Peergos - changing from a merkle-btree to a merkle-hamt (hash array mapped trie).

I’m using the champ (canonical hash array mapped prefix trie) variety of hamt. This has some super nice properties, including insertion order independence, never any rebalancing or splitting operations (which makes ipfs pins much faster), and tunable storage overhead and data churn. In the process I found and fixed a bug in the same structure in ipfs/filecoin.

The other interesting thing is that this data structure is amazingly young as far as data structures go. I think the original hamt was introduced in 2000, and the champ variation in 2015.

After doing a show and tell here on Lobste.rs for balística I’ve been trying to implement the suggestions I received for improvement. They were of varying degree of difficulty for me, so it’s still very much a work in progress. Thanks again to all those who took the time to look over my project and make suggestions!

Finally, the last big thing I want for balística right is now is graphs. While I’ve received a lot of help on how to do this in the form of comments like “use cairo” I still have no idea how to actually do use cairo to make a graph.

Hunting for remaining issues in Mitogen’s recently implemented fork support. Pretty upset to discover it’s somehow taking 8ms, that should be closer to 500 usec :S Sampling profilers are useless at this time scale. Currently thinking about some approach involving gdb’s reverse debugging or similar.. need to capture behaviour of 2 processes containing 2 threads each. After I’m done with that, using the new support to add better isolation for Ansible modules that leak state or monkeypatch stuff

I’ve been tinkering with my Elm-based static site generator which is based on elm-static-html. That library relies on Elm’s undocumented native modules feature which is going away with the next release of the compiler, so I’m trying to figure out if there’s a way to do Elm-to-HTML conversion without relying on native modules. 

• CompilerKit is a Swift library of data structures and algorithms used in compilers. It’s my learning project.

• Applying to OMSCS. I really hope they’ll take me without an undergrad in CS. Any advice/help is very welcome and greatly appreciated.

$WORK

• Last week I setup a basic repo for a react payment widget to integrate into client websites, so I’ll build it over this week and the next.
• We’re looking at oloo and their timesheet app to track hours spent on specific projects, ideally I would of preferred some kind of Taskwarrior + Timewarrior + Bugwarrior setup, but I havent found a package that makes it all easy to use for non technical staff..

$PERSONAL

• Just one chapter of The Rust Programming Book left!
• Still playing around with my email in acme, it’s really fun, probably going to start looking at git in acme next.
• I’d like to play around with Pijul, the rust vcs.
• Probably update to the latest openbsd snapshot in the beginning of the week, to help test 6.3 (I’m already running -current, but from a snapshot before they froze the tree)

Currently: Looking at learning Coffeescript for something silly. Looking online, ES6 should replace it but ES6=ES2015 so that’s only been around for at most 3 years, less if you count the time for browsers to become compatible. And isn’t CS still fewer brackets, both curly and round? Although I probably shouldn’t be getting distracted by any of this given all the other stuff going on…

Last week: Released guitktk, the simultanous GUI and toolkit maker.

This week:

• Improve guitktk’s speed. I can always “hand compile” function calls to their actual effect but this reduces polymorphism. What to do?
• Finish blog post for what all my github projects are about.

I’m working on Mozillas Cornice - I’ve already added Marshmallow schema validation, now I’ve added out-of-the-box API Explorer for cornice_swagger package.

I finished up my reverse engineering of QCRACK, the old Quake cracking program. I’ll be doing a full writeup to submit for something about how this was my first reverse engineering project.

We had a hackathon and another release for my little social network, pnut.io, over the weekend.

Now I’m organizing my thoughts about our next improvements, and helping developers implement the changes. It looks like we’re already set for GDPR. Not sure how I’m going to improve our search back end yet.

And if at all possible I’m going to feed the bees in between the rain!

This week will mostly be busy with job interviews, though I’d like to finish implementing an rtpMIDI backend implementation for MIDIMonster. Ideally with native support for the AppleMIDI session protocol, though the whole specification is…. not that good.

Other than that, my to-do list contains a whole host of items, including publishing a new article on my homepage that has been in the works for long enough now.

This is my last week as a Research Assistant so mostly wrapping things up and then leave for the easter weekend.

I have started to prepare the courses that I will take next semester. This mostly consists of fighting the latex compiler, getting the correct number of passes for reference resolution, etc. I am using latexmk for that and the process has provided me a lot of headaches so far but I have a setup that (mostly) works.

One of the last things for the winter semester is an essay about machine learning, today I am going to discuss the topic, which will be related to ethics and philosophy of science. I am really looking forward to that.

Working on cache server nuster, fixed a security bug which can bypass ACL check, implementing cache stats functionality.

When cache stats is done, nuster would be a full functional cache server. After that, I will start to refactor some code(mainly rename cache related variables/functions with same name used in haproxy v1.8), and import haproxy 1.8.

Following on from last week I’m pushing ahead with getting to 1.0 releases of at least three packages that are pretty close to being done:

• Koalephant Certificate Manager: Simplifies use of TLS certificates (primarily with HAProxy and Certbot): builds joined files, fetches OCSP stapling data, syncs certs to peer proxy nodes.

• Koalephant Proxy Manager: Simplifies config of HAProxy (primarily in a multi-node cluster): builds a modular config using the ‘conf-available/conf-enabled’ method, sets up Peer lists and syncs cluster config/enabled configs to peer nodes in the cluster.

• Koalephant Failover Manager: Simplifies use of Keepalived and VRRP to add IP failover to a HAProxy cluster: builds a Keepalived config, sets sysctl parameters as required, and runs hooks on failover status.

If those all go well, I’ll be pushing into a PXC manager, again, to simplify setup, sync config/etc in a Percona Cluster

After a bit of a hiatus I managed to get my next debugging book review up, and I’m on to the next set of the books. The next few sets of books look like they’re going to be a lot more fun than the last set.

This past review was a pain because of some life stuff getting in the way, but also the books were big, dense, and one of them was really, really bad. Turns out writing a fair review of a bad book is a lot harder than I thought.

I also added an RSS feed to my site and migrated the site to HTTPS hosting.

Now that all my exams are finished, I’ve been thinking about a small project to work on until university starts again. One thing that really annoys me is how information is dispersed and among different websites, which all take a while to load, and an even longer while to find what one actually wants. But it’s all “stupid” algorithmic querying of information, and it’s always the same (when to take the train, where to go, what’s the weather, what’s on my agenda, what’s in the news, etc.). Some people might make an app, but since I want to try using my phone less, I thought about using a thermal printer (or maybe a smaller one) with a raspberry pi I have lying around to create a daily “summary” of sorts, printed out on a receipt.

So I’ve been looking into tools to parse RSS, the API for my local railway network, mechanisms to communicate between my org-mode agenda and the raspberry pi and of course I’m also trying to find out what adafruit’s API allows (one can print via lpr so maybe groff could be used…). The cleverest way would be probably to package it all into a shell script and call it (only when necessary via cron). Maybe I could also create a web copy, for links and other resources, in case I want more information?

I’m working on three things atm (maybe 4).

First, I’m tinkering with SECL again, I want to extract as much of the lexer and parser as possible so I can reuse the entire construct for different usage scenarios (HTML templating in Go for example)

Second, I’m working on a mastodon service like tweetlonger (I think that is how they called it) to allow people to post blogposts with their toots. I’m also chewing on a federated reddit variant based on this using activitypub as a common protocol.

Lastly, I’m trying to work out how to best wire up a second RGBW strip to the current one across the room and some at the ceiling. I need about 20 Watts of power at 5V plus communication stuff. I was thinking about handrolling a custom 1-wire protocol, the arduino is way to busy for normal 1-wire, I need to significantly lengthen timings (atm I can, according to my math, squeeze about 20 bit/s over the wire). Though I might just scrap it too if it turns out I can’t do it without needing a thick wire or high voltage >50VDC.

I have continued to hack on my fork of ISETL. I have replaced many macros with inline functions, added nullability annotations, and gotten it to compile cleanly as either C11 or C++14.

Comedy! I have some shows to edit and do post on (audio work, heck yes), I did some workshopping yesterday and will be at an open mic tonight.

Meetup presentation! I am working on a presentation for a local code club meetup some time in the summer. I’m tryin to keep the topic under wraps but it’ll have to be about working culture and not really about code.

Magento 2! All I do these days is slowly port our proprietary Magento 1 extensions in Magento 2.

I translated the navy seals copypasta into Brithenig.

My personal/toy project this week is writing something that sorts video frames based on their similarity, then recreates the video so that each frame is followed by the nearest remaining frame (with the audio synced as well). I have it written, but the current implementation is extremely slow.

Gonna spend most of the week learning about Elixir. We already used it in a Distrubuted Algorithms class at my college, but it was interesting enough to pique my interest even more.

Apart from this, need to write more for an OSDI submission.