Spent a lot of time polishing some of my libraries to get them to 1.0 (with help from others).
Rewrote my CSV library in Rust. Pretty sure it’s one of the fastest general purpose CSV parsers. It’s at 1.0-beta. Close to a 1.0 release.
I managed to publish 1 blog post (on aforementioned CSV library), which keeps my pace going.
Helped with the SIMD stabilization effort for Rust, but others have helped a ton too. We basically crowd-sourced writing implementations for all the various Intel vendor intrinsics (there are ~thousands) and it worked really well!
Wrote a small CLI data entry tool to import data into a ledger format. I still need to copy & paste tables from PDF files though on occasion. /sigh
Spent a significant amount of time working on a low level information retrieval (think Lucene, but one layer down) library. I have nothing to show for it so far. It’s hard.
Began a giant refactor of Rust’s regex library internals that will probably take years. This year I almost finished a rewrite of the parser. The goal of the refactor is to make it easier to add optimizations and break apart the internals. A secondary goal is to support stream matching. I don’t know if I’ll succeed in either endeavor!
I started work on a principled approach (read: not some random mish mash of Python scripts) to parsing the Unicode Character Database, with the intent of using it to (mostly) provide complete support for level 1 Unicode support. I’m also casually building a tool to browse/search the UCD. I had been using a Go tool that someone else wrote, but it’s… surprisingly slow.
Continued active maintenance of ripgrep. Interestingly, I didn’t add any new major features this year (perhaps UTF-16 support qualifies, but in terms of code changes, it was very small). I do still have some planned that I had hoped to get to this year, but that ain’t happening. :)
I discovered Dan Carlin and listened to all of his Hardcore History podcasts during my commute.
At $work, I did lots of stuff too. A big part of this year was spent adding access control to our system. I’m also working on a collaborative graph editing system (think Google Docs but for knowledge graphs) which has a ton of interesting problems in it.
Oooh, I’d love to hear more about this data retrieval library. What formats do you intend to support? Been looking for something more lightweight then lucenne.
Hmmm. I don’t know what you mean by that. It has its own format, just like Lucene has its own format. It isn’t going to support Lucene’s format is that’s what you mean.
The idea it’s to provide a library for creating and querying a single IR index. In Lucene terms, this is at the level of a single segment. It uses fsts for the term index, uses skip lists for the postings and all that. It is mostly just a prototype at this point. I’m struggling with what the API should look like. For example, you invariably need to talk about multiple segments for merging, even though I was trying to avoid that (merging needs knowledge of the format on disk to be fast). This in turn means making assumptions about identifiers. So it is a bit of a knot at the moment.
I ended up writing a ClojureScript micro-framework for Node called Macchiato. I primarily work with Clojure, and I found a few cases where the JVM is not the best fit. The two main limitations being relatively high memory requirements and startup time. This makes the JVM a poor fit for things like AWS Lambda. I had a use case for making some lightweight services at work, and wanted to learn a bit about the Node ecosystem, so this was the result.
Got WinWorld, a site for vintage computer software preservation, back online after some drama caused bus factor to drop to 0, and had to rewrite the codebase, this time making it free software, so this shouldn’t happen again.
Wrote a music player because everything else sucked, let’s NIH it. Turns out a friend or two liked it too!
My seemingly vain efforts to get Mono working on Haiku, everyone’s favourite BeOS clone. Every time something gets done and we get closer, but never get finished.
Work, moving to a new house, my older son starting kindergarten, my younger son starting preschool, and various classes meant I didn’t have as much time to work on personal projects this year as I would’ve liked to.
At work I completely rewrote the packet capture layer of our product; about 5000 lines of C got us capturing 20Gbps with no packet loss on commodity (though beefy) hardware, and another layer on top of that got stream reassembly and file extraction for all HTTP, HTTP/2, and SMTP streams with less than a minute lag. That was fun.
The only personal project that really got any traction this past year was mtm, a terminal multiplexer in ~1000 lines of C.
I’ve worked on a variety of improvements to Pony. The improvements have ranged from documentation, to CI process, to adding generalized runtime backpressure to Pony’s actor scheduling. That’s my “for fun” work that I do mostly on weekends.
My day job is working on Wallaroo. A stream processing engine (with Python and Go language bindings) that’s written in Pony. This year, we open sourced (under an open core model) Wallaroo.
It’s good you open-sourced it since that might benefit Pony developers still trying to learn how to use the language for real-world apps. Tutorials and examples only go so far.
I also commented in the HN thread but here will be a little more personal. The main thing I worked on in 2017 was taking xi editor from being promising to almost something you’d want to use for your daily work. I also took 3 months off to do a batch at Recurse Center, which was great fun (crossword authoring, loading data through the Apple 2 cassette port, and logic). I somehow managed to interleave that with working on the Rust infrastructure for Fuchsia, which might well be the work with the most lasting impact.
For 2018, I’m planning to focus more on xi (I realize that too much multitasking is one of my weaknesses), and think there’s a good chance we (the community and I) can make it something really good and really usable.
My newborn daughter took most of my time as I took my parental leave from June.
I continued rework of structures of my graphical mouse driven text editor werf under a separate project that was supposed to compare performance of different text editor structures on real life editing patterns, taken from the most edited file in Linux kernel git repository - text-vs. I finished the text editing routines, but did not yet integrate it back. It’s C, Xlib, cairo and fontconfig.
I started more projects or experiments:
Some kind of a game with learning of modern OpenGL in mind. Nothing to see here. C, SDL 2, OpenGL 3.3.
2d path tracer - single square hard coded light source and that’s it. Maybe calling it a path tracer is too much of a stretch, but it’s a thing that I wanted to explore for years. C and SDL 1.2. Nothing to see here either.
Tomatoaster a ChromeOS-architecture like Linux distribution. I’m mainly interested in replicating atomic updates using two read only rootfs partitions. I’m using Void Linux packages and build system to make a rootfs image. Along the development I did a few pull requests to the Void Linux project which was a first for me. My script can build an image that is runnable under qemu on modern kernels without root privileges. It’s not a particularly interesting image, but it’s able to just run Firefox in bare X session. Next I want to do a proper image with boot loader.
Dumbenchmark - a benchmark that I wanted to use to check the performance of several VPS providers. Measuring compile time of certain Linux kernel. Also indexing of sources and full text search through built index with SQLite FTS5. The last part was kind of exploratory for my search engine project that will index only pages without ads. Shell and C.
I also did exploration in the web dev sphere. I did those on my Chromebook so I did not manage to push them to any repositories yet. One is a continuous calendar with free form editing. The other is a toy that uses web workers to do a poor man’s version of Unix pipes - it did not work on Firefox, because of some inexplicable reason.
I should probably stick with one and finish it. Maybe getting some kind of help would be useful. Will see how it goes in the new year.
Designed and built a DNA analysis pipeline system using RoR for state/pipeline management and Kubernetes for application orchestration.
Designed and built a referral management platform w/JS widgets and API for integration options. (A referral is like, you refer another user and you get a month free).
Designed and built a simulation system for modeling nuclear proliferation risk.
Designed and rolled out a software engineering process to help a smallish (about 10) inexperienced group of devs do things in a more repeatable, predictable, and visible way.
Started stabilizing and modernizing an old codebase.
A twitter bot that brings me small amounts of joy in semi-regular intervals: @NYT_first_said
It’s nice having code running that you can interact with every day, lends itself to lots of tiny improvements.
I also made a lot of visual code sketches, and honed my tooling and process for iterating and version controlling them! I hope to expand on these tools and make them useful for outside consumption this year.
https://maxbittker.com/ Built a site, went very slightly overboard, and listed some art and blog posts on it!
Working on an open source product (Sentry) was also really fun! Looking forward to 2018 there.
finalized the design, certified and pushed into production the best corded telephone handset since Alexander Graham Bell. Very likely destined to be the last of its kind, too.
tons of software and sometimes hardware customization of the products for various industrial customers.
started on developing an entirely novel kind of product; it is really pushing my technical skills to extreme.
My SO’s car (as well as her mother’s, her brother’s, her other brother’s, and my coworkers’). It has been an unending source of headaches and busted knuckles. I will be continuing this labour well into he forseeable future. It is a labour that I primarily carry out in anger.
I learned a lot about small engine repair, but only one thing about myself: I will never own an automobile newer than the one I currently own.
Left my old job doing line-of-business Java stuff for satellite communications companies and found a new one doing machine learning software for circuit designers. Quite enjoying myself so far.
Took over maintenance and development of a statistics-driven benchmarking library in Rust called Criterion.rs. It’s a great tool but it was never officially released until I got to it a few months ago. I’ve learned a lot about community management and open source, partly by trial and error and partly by asking other project maintainers.
Work on a bunch of side projects, sad to say none of them amount to anything at the moment. New year’s resolution is to change that and actually release a project or two this year.
Got my reading game on, buying a bunch of books in the last quarter of the year and reading a couple. Deep Work, half of Clean Code (almost to the bit where they rewrite stuff “correctly”), The Leprechauns of Software Engineering, and now reading Getting Things Done.
Moved to Switzerland from Amsterdam, after previous year having moved to Amsterdam from Mexico. Plan to stay here a while.
Published a library that transparently handles standard directories for applications (config dir, cache dir, etc.) across Linux, Windows and macOS. Implemented the library in Java and Rust.
Wrote some articles on language design, and lessons learned from mistakes.
ledgerformat. I still need to copy & paste tables from PDF files though on occasion. /sighunsafe.At $work, I did lots of stuff too. A big part of this year was spent adding access control to our system. I’m also working on a collaborative graph editing system (think Google Docs but for knowledge graphs) which has a ton of interesting problems in it.
Oooh, I’d love to hear more about this data retrieval library. What formats do you intend to support? Been looking for something more lightweight then lucenne.
Hmmm. I don’t know what you mean by that. It has its own format, just like Lucene has its own format. It isn’t going to support Lucene’s format is that’s what you mean.
The idea it’s to provide a library for creating and querying a single IR index. In Lucene terms, this is at the level of a single segment. It uses fsts for the term index, uses skip lists for the postings and all that. It is mostly just a prototype at this point. I’m struggling with what the API should look like. For example, you invariably need to talk about multiple segments for merging, even though I was trying to avoid that (merging needs knowledge of the format on disk to be fast). This in turn means making assumptions about identifiers. So it is a bit of a knot at the moment.
What I meant was something more lightweight resource wise then running a jvm.
I ended up writing a ClojureScript micro-framework for Node called Macchiato. I primarily work with Clojure, and I found a few cases where the JVM is not the best fit. The two main limitations being relatively high memory requirements and startup time. This makes the JVM a poor fit for things like AWS Lambda. I had a use case for making some lightweight services at work, and wanted to learn a bit about the Node ecosystem, so this was the result.
Very excited about new developments like shadow-cljs and macchiato in the clojurescript world! thanks Yogthos :)
Thanks, 2018 should be a pretty exciting year for ClojureScript. :)
I did not write a single line of code for money all year. It was a good year.
What did you do instead? Management, consulting, bartending?
Got WinWorld, a site for vintage computer software preservation, back online after some drama caused bus factor to drop to 0, and had to rewrite the codebase, this time making it free software, so this shouldn’t happen again.
Wrote a music player because everything else sucked, let’s NIH it. Turns out a friend or two liked it too!
Helped my friend with his IRCd.
My seemingly vain efforts to get Mono working on Haiku, everyone’s favourite BeOS clone. Every time something gets done and we get closer, but never get finished.
Work, moving to a new house, my older son starting kindergarten, my younger son starting preschool, and various classes meant I didn’t have as much time to work on personal projects this year as I would’ve liked to.
At work I completely rewrote the packet capture layer of our product; about 5000 lines of C got us capturing 20Gbps with no packet loss on commodity (though beefy) hardware, and another layer on top of that got stream reassembly and file extraction for all HTTP, HTTP/2, and SMTP streams with less than a minute lag. That was fun.
The only personal project that really got any traction this past year was mtm, a terminal multiplexer in ~1000 lines of C.
I’ve worked on a variety of improvements to Pony. The improvements have ranged from documentation, to CI process, to adding generalized runtime backpressure to Pony’s actor scheduling. That’s my “for fun” work that I do mostly on weekends.
My day job is working on Wallaroo. A stream processing engine (with Python and Go language bindings) that’s written in Pony. This year, we open sourced (under an open core model) Wallaroo.
It’s been a wild and crazy year.
It’s good you open-sourced it since that might benefit Pony developers still trying to learn how to use the language for real-world apps. Tutorials and examples only go so far.
[Comment removed by author]
They were in fact classification problems.
You can rent GPUs from EC2 for minimal upfront capital. It does get expensive pretty quickly, though.
I also commented in the HN thread but here will be a little more personal. The main thing I worked on in 2017 was taking xi editor from being promising to almost something you’d want to use for your daily work. I also took 3 months off to do a batch at Recurse Center, which was great fun (crossword authoring, loading data through the Apple 2 cassette port, and logic). I somehow managed to interleave that with working on the Rust infrastructure for Fuchsia, which might well be the work with the most lasting impact.
For 2018, I’m planning to focus more on xi (I realize that too much multitasking is one of my weaknesses), and think there’s a good chance we (the community and I) can make it something really good and really usable.
My newborn daughter took most of my time as I took my parental leave from June.
I continued rework of structures of my graphical mouse driven text editor werf under a separate project that was supposed to compare performance of different text editor structures on real life editing patterns, taken from the most edited file in Linux kernel git repository - text-vs. I finished the text editing routines, but did not yet integrate it back. It’s C, Xlib, cairo and fontconfig.
I started more projects or experiments:
Some kind of a game with learning of modern OpenGL in mind. Nothing to see here. C, SDL 2, OpenGL 3.3.
2d path tracer - single square hard coded light source and that’s it. Maybe calling it a path tracer is too much of a stretch, but it’s a thing that I wanted to explore for years. C and SDL 1.2. Nothing to see here either.
Tomatoaster a ChromeOS-architecture like Linux distribution. I’m mainly interested in replicating atomic updates using two read only rootfs partitions. I’m using Void Linux packages and build system to make a rootfs image. Along the development I did a few pull requests to the Void Linux project which was a first for me. My script can build an image that is runnable under qemu on modern kernels without root privileges. It’s not a particularly interesting image, but it’s able to just run Firefox in bare X session. Next I want to do a proper image with boot loader.
Dumbenchmark - a benchmark that I wanted to use to check the performance of several VPS providers. Measuring compile time of certain Linux kernel. Also indexing of sources and full text search through built index with SQLite FTS5. The last part was kind of exploratory for my search engine project that will index only pages without ads. Shell and C.
I also did exploration in the web dev sphere. I did those on my Chromebook so I did not manage to push them to any repositories yet. One is a continuous calendar with free form editing. The other is a toy that uses web workers to do a poor man’s version of Unix pipes - it did not work on Firefox, because of some inexplicable reason.
I should probably stick with one and finish it. Maybe getting some kind of help would be useful. Will see how it goes in the new year.
A twitter bot that brings me small amounts of joy in semi-regular intervals: @NYT_first_said
It’s nice having code running that you can interact with every day, lends itself to lots of tiny improvements.
I also made a lot of visual code sketches, and honed my tooling and process for iterating and version controlling them! I hope to expand on these tools and make them useful for outside consumption this year.
https://maxbittker.com/ Built a site, went very slightly overboard, and listed some art and blog posts on it!
Working on an open source product (Sentry) was also really fun! Looking forward to 2018 there.
Let’s see.
At $work
Hobby
My SO’s car (as well as her mother’s, her brother’s, her other brother’s, and my coworkers’). It has been an unending source of headaches and busted knuckles. I will be continuing this labour well into he forseeable future. It is a labour that I primarily carry out in anger.
I learned a lot about small engine repair, but only one thing about myself: I will never own an automobile newer than the one I currently own.
I also finally moved from Canada to the U.S., which has been a goal of mine since 2014.
Welcome to America! I hope you’re enjoying what might be your first New Years here. :)
Thanks! I actually fell asleep at 8pm and the San Francisco fireworks scared me awake at midnight.
You’ll get used to it haha
- Added a web playground for PISC
- Increased performance of PISC (mostly early days de-stupiding):
-
- Removing regular expressions from the hot-path of the code (big memory reduction)
-
- Caching name-lookup results for words, especially for numbers (2x faster interpreter)
-
- Removing an error catching
- Made dicts much more ergonomic to use by building a nice set of prefix words for them
- Added a test suite to PISC that has caught at least 3 regressions in refactoring work
New job at Greenshades Software New Tech/Learningdeferfrom the hot-path of the PISC interpreter (2x faster as well)- Learned enough Org-mode in Emacs to use it for tracking work related information
- Learned the basics of Magit, though it’s still too slow for Windows
- Set up Weechat, SyncThing, Hugo and Linx on my personal server
- Added a very basic GUI for approving/deleting comments to the CMS for junglecoder.com.
Game Jams:- Ludum Dare 38: Bullet Surfing: A Retro/Vector game about collecting bullets
- Alakajam 1: Caster Fight: An overly simplistic fighting game that marks my first proper foray into pixel/character animation
Books-
-
-
Audio BooksThe Art of Procrastination
The Phoenix Project
Purchased the Humble Bundle “Be a Coder” pack,
which I’m going to try to read through and possibly write about during 2018
I wrote a book about working with time and temporal data in PostgreSQL.
Firefox for iOS.
Work on a bunch of side projects, sad to say none of them amount to anything at the moment. New year’s resolution is to change that and actually release a project or two this year.
Got my reading game on, buying a bunch of books in the last quarter of the year and reading a couple. Deep Work, half of Clean Code (almost to the bit where they rewrite stuff “correctly”), The Leprechauns of Software Engineering, and now reading Getting Things Done.
Moved to Switzerland from Amsterdam, after previous year having moved to Amsterdam from Mexico. Plan to stay here a while.
Published a library that transparently handles standard directories for applications (config dir, cache dir, etc.) across Linux, Windows and macOS. Implemented the library in Java and Rust.
Wrote some articles on language design, and lessons learned from mistakes.