This is the weekly thread to discuss what you have done recently and are working on this week.
Please be descriptive and don’t hesitate to champion your accomplishments or ask for help, advice or other guidance.
Last week, I built a barn, set up the hops separator, and started troubleshooting an industrial ERV unit that we’re hoping will assist with the process of dehydration for our orders of dry hops. Also we turned some cattle in on the mostly-unused pasture next to the hopyard!
This week, I’m hoping to knock the harvest out as quickly as possible, so that I can then return home to KY when the rest of the hands leave, or at least start in on the myriad other projects the farmer has waiting. There’s nothing really stopping me from leaving just after the harvest, but I don’t want to leave him hanging, either. If I’m not leaving, then I’ll probably start in building another two barns, grading gravel (in preparation for pouring a concrete patio), and so on.
If I do leave early, I have a small fence to put up, a pig hut to paint and move, a chicken coop and tractor in need of repair/moving, swales to dig, a larger fence to put up, and I’d love to begin tearing into and renovating the den.
That sounds so much more tangibly satisfying than software work. What is a swale?
Think contoured ditch for directing runoff water. We’re planning to put them in to guide runoff towards what will be a plethora of annuals and perrenials, which’ll hopefully take care of feeding the animals (mostly).
Negotiating among new gigs, hope to have news next week. If anyone knows a California lawyer familiar with IP assignment contracts, I’d love a referral.
Also getting a CopperheadOS phone set up. It’s an enraging tinkertoy of a computer even without Google’s apps installed, we’ve built a pretty crappy, closed future. The Android backup story is especially ridiculous… either you give your data to Google and hope they give it back, turn on a debug mode to run “adb backup”, which Google sabotaged to ignore much of your data, or download a rootkit from a sketchy website with lots of warnings about bricking your phone and then buy some commercial software. A pocket supercomputer can’t be backed up as thoroughly, securely, or easily as a 1983 IBM XT.
My first computer was a used IBM XT that I bought in 1987! All my data from it is extremely secure, on degaussed floppies in a landfill somewhere.
I’ve been considering getting a CopperheadOS phone, glad to see a fellow lobster blaze the trail! The other option I’m considering is Replicant, although I would be restricted to an older model phone.
I’m having a slower week due to travel again. This time, we are just driving around a bit in Thailand. This turned a bit problematic when our first car had a problem on the first day. We resolved this by getting our other car from home and resuming the trip. We’ll pick up the first car from a repair shop on the way home.
We did get to see a fairly large Asian Water Monitor next to our room on Sunday morning.
Apart from that, I’m still working on my initial implementation of some RDF stuff in Rust and am writing up a lot of documents about what I’m working on in the bigger picture … I’m looking forward to talking about that more publicly in the coming weeks.
I’ve had someone working for me for the last week and that’s going well. I’m working on bringing on 2 others in the coming days.
Enjoy your week!
I’m basically convinced you actually post from Jurassic Park.
This week is kinda fun, and hopefully won’t trainwreck.
I’m doing a 3-day contract + optional 2-day cleanup on a chatbot, filling in for some folks who need a hired gun. I had sworn that I’d never do backend development again unless it was in Elixir/Erlang, but damned if NodeJS doesn’t keep pulling me back in. :(
This weekend past I started on a simple prototype of an AR thingy-as-a-service, written in Elixir. If anybody wants to email their current gripes with mobile AR integrations, I’d appreciate it. The folks I’m working with seem pretty squared away and communication and brainstorming was quick and reasonable; we even wrote some of the damned docs out during discovery before laying down code, which was mutually appreciated.
I’m working on adding generalized backpressure to Pony. In particular, informing an actor when it sends to an actor with a loaded queue so that at the application level, action could be taken beyond what can be done automatically at the runtime level.
I spent some time thinking about this before (at least for now) abandoning Pony.
My one good idea was for the receiving actor to be able to set arbitrary watermarks on its message queue, and for it to be able to associate arbitrary chained functions to any attempt to message the actor when the watermarks were succeeded.
This would put the logic for the backpressure in (correctly, in my belief) the receiver, and also allow customizable backpressure modes (e.g. at 1000 queue, take 100 reductions from caller; at 10,000 queue, take 1m reductions from caller; at 100,000 queue, take all reductions from caller, return warning, ignore message, shed load…). Such a thing would also neatly work with distributed pony were a thing such as that ever to exist.
We can’t do it based on the size of the queue as we don’t know the size of the queue (and can’t for performance reasons).
Roughly, what we are doing is to make a queue as loaded if during a scheduler batch run, it was unable to run for the full batch size without running out of messages to process. If you send to a loaded queue, you get deprioritized by the scheduler, thereby hopefully allowing the consumer to catch up.
The work I’m doing is to indicate to user code when they are sending to a loaded queue so they can be more proactive.
I feel like having a U32 in the actor header caching the length of the message queue would not be unperformant, especially if it were, e.g., in or a replacement for the actual head of the message queue and so likely in the same cache line.
Having access to that length would be super great, though, because (a) operability and debuggability a la erlang, and (b) possibility of mitigating more kinds of scheduler collapse (algorithm can understand how bad the problem is) in a cleaner way. Anyway, just my thoughts, hope the algo works out! Pony’s definitely on my list of things to go back and look at one day.
I started writing a Docker Registry using Servant over the weekend. The routes are spec’d out but need to be updated with appropriate header responses, etc. Pushed it to github because of this thread, so now it’s off my laptop and in the real word.
My GraphQL/Relay static site generator is coming along really well. There’s some image manipulation things I want to do because the images as they exist today are optimized to my liking. This post, for example, has an image that is way too heavy for the page and I want to cut it down into thumbnails and headers automatically in the build process. More importantly perhaps, I need to document how it works and then how to develop plugins for it. I also want to make setting up a CI process trivial, so will likely put some work into that this week.
This past week, and this coming week too I expect, I’ll be polishing the edges on, and pushing to production my new website (dev site, source code). I re-wrote it using App Engine since I figured it was finally time to ditch virtual servers and use a service such as Google Cloud. The new version is much more testable, scalable (not that it’s required) and has lots of cool metrics supplied by Google that I didn’t have before, so I’m happy with the migration.
I also got my reading for next year at university (Programming in Prolog and some 100 pages of maths notes), so I am getting up early in the morning to go through it. Reading technical stuff after work is hard due to a lack of mental energy, so getting up early is the only option. I’m also trying to fit in preparation for exit interviews from my internship at the same time, figuring out what to do when is quite the balancing act!
Finally, I will be finishing my student loan applications for the coming year, which as of now I can safely say have been the most bureaucratic undertaking that I’ve ever had the misfortune to experience. The annoying thing, is that if I happen to make a mistake, then the cycle of the mistake being flagged, the application being sent back, me fixing it and sending it again will take 2+ weeks and mean that I will likely not have the money on time to start the course.
I am trying to turn our basic fluentd based log aggregation and collection system into something that’s high availability, resilient and fault tolerant so we can start to lean on it more heavily for our data warehousing systems. It’s a lot of fun - but distributed systems are hard!
Keep in mind that it’s easy to overwhelm a log system, and that syslog is lossy by design, much for this reason. Fluentd isn’t syslog, for sure, but if you have any chance at reliable, non-lossy logging, you’ll need to be able to spool to disk, which becomes a potential reliability problem in a cloud environment. Good luck!
Getting around to finishing up some work on my floodfill.js library. Basic code for v2 is done but needs more tests and documentation. It’s funny how this evolved, from a gist I posted in 2011 to a request for a library, to folks actually using it and pointing out bugs and requesting features. If I have time I will make a github-page that has a demo. Here is the dev branch for those who are interested - https://github.com/binarymax/floodfill.js/tree/development
Aside from that I’m finding time to write up some procgen explosions for my book. I’m knee deep in animation techniques so is quite interesting to explain. Here is a poor GIF of an example in action, using exponential interpolation: https://mobile.twitter.com/binarymax/status/760786236684562432/photo/1
All that leaves is the usual 50 hours of my awesome full time job :)
Building a postgres database, remembering a few db concepts I had forgotten. Finishing a website contract and setting up a few hakyll generators to make translating websites much simpler. (Well simpler for the people writting the content)
Looking for new work and trying to clean up somebody elses' cmake mess. I promise you the two aren’t related.
This week I am working on a bytecode stack-based runtime for my experimental programming language Icarus lang, last week I wrote a good chunk of the datastructures I will need, so this week I am looking at writing the actual ‘bytecode compiler’ for a non-turing complete subset of the language.
So far this has been fun as I have never written code to convert from imperative code to a stack-based machine, I have only ever used stack machines for stack languages, my first-pass algorithm can be seen as a comment here.
Last week on Icarus I was able to get some simple code snippets (non-turing complete subset) to compile to C and run ‘cleanly’, the most interesting one I have written so far is
fn get_str(name::String) -> String
return concat(concat("Hello there ", name), ", very nice to meet you")
fn main() -> Void
Hello there Jennifer, very nice to meet you
If you’re interested in the full process (which shows both transformation and the compiled C) then you can see this in my README here
So this week’s goal is try make progress on having the same subset also compile to my stack bytecode and then write the runtime for that to actually execute.
I have some small documentation patches to Pony lang that I want to finish off.
I’m also hoping to find some time to work on a tool written in Carp lang as a test of the language, doing so will also require me adding a bunch of core library features for dealing with filesystems - this should make Carp appropriate for scripting applications.
It is likely the Carp work will slip a week or two…
Last week I also met with a local university programming language research group to discuss me potentially starting a part-time MSc. Computer Science next year, I spoke to some of them about potential projects, so this week I will be reading into one project in particular and hopefully scheduling another meetup.
All of the above is in my spare time, so it all depends how busy work keeps me, I expect ~ half of this to slip until next week.
As for actual work I am working on migrating our team’s oncall rotation management from the current pile-of-cards-and-glue solution to something more sane.
I am going to try to publish another hifibyapg this week with another interview. The last 2 were pretty successful, and I think this, too, will be interesting.
At work, I am in post launch cleanup mode for a feature we just shipped. And, also starting to think about the design and requirements for the next big piece of fun.
Additionally, I have a ton of reading to do–I have about 4 different books in progress, so, I hope to find time to make progress on at least one or 2 of them.
Finally, I’d love to prototype a system I have been thinking about so I can explore the space a bit. It doesn’t sound too exciting, but I’ve been wondering what a robust job queue system done almost completely server side looks like, with retry logic, timeouts, etc being something you just get. Nothing concrete, but on my mind, nonetheless.
Last week I released the Android app for my bookmarking service for developers, Larder, so this week I’ve been doing some little tweaks and bug fixes based on dogfooding it. There’s still a way to go (no edit or delete from the app yet) but as a way to save bookmarks via the share menu, and for browsing and searching bookmarks, it works pretty well. I mostly do web dev so it’s nice to feel like I can finally call myself a decent Android developer as well.
Assets for a board game, a novel, a bunch of blog posts for various things, and refactoring some old React code that needs tidying up on a small web app that logs mankind’s more interesting activities from 1750 to today.
Despite trying to avoid web programming for years now, my job has forced me to learn PHP and JQuery. So after spending half of last week on it, I’ll undoubted spend the rest of this week with it as well.
For moneys, I’m gonna be recording videos about our infrastructure and support tools to help newcomers join the team. I’ve also been asked to compare and contrast Kinesis and Kafka, which are the exact same thing as far as I can tell, and figure out whether they fulfill whatever need we need them to.
For not moneys, not sure. Vacations next week, so, probably a good deal of planning those. I should also be getting the answer for my offer on the house at the latest this week, hopefully that translates into an accepted offer. (And then I can run around to find an inspector and a mortgage loan, and set a date with a notary in December.)
Kafka and Kinesis are quite different once you get into the details.
Costs wise, it’s also a different way of thinking.
If you happen to publish that comparison, I’d love to read it!
Yeah I figured there were some differences, I don’t know if I’ll get as much time as I’d like to do an in-depth comparison. I think they kinda wanna use it as some sort of perma-storage for data; I know for sure that doesn’t work with Kinesis (up to 7 days retention maximum), and by default Kafka has 7 days of retention too. At the end of the day I think I’m going to be able to say “Well, both can process incredible amounts of data, and you can tweak your retention policy for a certain amount of time so that you could potentially seek some distance in the past, but it’s not reasonable for data storage”, or something like that. Or I’m missing a fundamental point in what they’re asking me, or in the tech that I’m looking at.
If I get enough stuff to write about, then, yeah I guess maybe I could do a small writeup. If that happens I’ll post it here.
!Job: Expanding my Docker materials into a 3-day training workshop – mostly focusing on the project/problem sets.
Job: Writing up an internal doc on app configuration in a cloud environment (from env vars to etcd to vault). Last, but not least, trying to help people understand that testing != monitoring, and that monitoring/metrics/operability need to be a first-class features and not afterthoughts.
I’m continuing to learn a lot about node.js and mongodb as part of my work for a client, which continues to be an illuminative object lesson in why we’ve created software engineering. Mongodb’s nature is causing a lot of what I would, as a long term relational guy, consider as unnecessary rework, backtracing, and bug hunting, as well as intentional misdesign to work around the latter. This can be pretty frustrating!
In my off time, I’m working on setting up a Kubernetes cluster and putting it through its paces. Docker-as-Docker never impressed me much, but Kubernetes fills in most of the important missing operational details, and Kelsey Hightower’s posts and videos are insightful, entertaining and well-put. My medium-to-long-term goal is to see if I can bring up a scaling cluster of erlang releases and quantify their performance at $TASK relative to the bare-metal case, and then check out the relative deployability, debuggability and crisis management facilities. So far I’m impressed with it, which is probably not surprising given its parentage.
In other news, I continue to recommend to all developers the practice of making homemade artisan bread, with http://cooking.nytimes.com/recipes/11376-no-knead-bread as a foundation and using https://www.amazon.com/Lodge-LCC3-Pre-Seasoned-Cast-Iron-3-2-Quart/dp/B0009JKG9M as the equipment. You learn a lot about your patience levels, your understanding, and the complications of the real world by failing a lot at a physical task that looks, on the surface, trivial. Humility: learn some.
Bug fixing and preparation for code release mostly. Making notes about future features and refactoring. Experimenting with embedded Lua.
Job: Fixing bugs for our search infra, add new features, especially an implicit filter for category
Hobby: This week I’m creating a Python interface for Facebook fastText
Fiddling with FreeBSD (mostly writing scripts to make my dmenu launcher smarter), and working a bit on sanegl. Last week I didn’t fully finish adding image based lighting like I wanted to. I did refactor some of the texturing code a bit though, so some progress was made. Planning on adding IBL and shadows this week.
will probably add a few more small pieces to my z machine interpreter. it’s a very relaxing endeavour - all you need to do is adhere to the spec, and if there’s ever something you don’t understand there are dozens of other implementations out there to explain concretely what the spec was talking about.