1. 29

This is the weekly thread to discuss what you have done recently and are working on this week.

Please be descriptive and don’t hesitate to champion your accomplishments or ask for help, advice or other guidance.

    1. 12

      More optimizations to Rust’s regex library. I’m thinking about how to use ideas from grep to speed up general regex searching. For example, given a regex like \s+foobar\d+, how can we use the literal in the middle? grep does it by taking advantage of line oriented search, but you can’t do that in the general case. It seems like it should be possible to split the regex into three pieces: the first \s+, the literal foobar and the last \d+. Scan for the literal foobar quickly, then match \s+ in reverse from the start of the match and \d+ forwards from the end of the match.

    2. 8

      Work: Laying off what’s left of my already withered dev/devops team. Picking up the slack. Triage. Fighting burn-out in the face of the burn-rate.

      Personal: Finding my next job. Installing FreeBSD 10.3 in a VM before updating my personal VPS. MacPorts patches. Messing around with the Grav CMS. Cross-training for a bicycle half-century later this month and a century next month.

      1. 1

        That sounds unpleasant :-(

      2. 1

        Eek. Sorry to hear about that–if you need someone to vent to, hit me up on email or PM.

    3. 7

      Installed OpenBSD on my main machine. Also probably going to install it on my server.

      Continuing learning Common Lisp.

      1. 3

        and 4 rasp pis

        1. 0


      2. 1

        Oh god not you too.

    4. 7

      Working on my multiplexing keyboard (should be able to be plugged into multiple computers, and switch between them, but also act as a usb hub for devices like mice to switch with the keyboard and data drives to act as a netshare between all connected computers). The design is split/ortholinear but each module has a microcontroller sitting in it so you can use as many as you want. I finished the file for cutting out the faceplate (https://gist.github.com/distransient/cbd4c9366d8084d01b4ca5e24182c00d) and now I’m working on figuring out what material I’m going to use to prototype with and what controllers I want to source.

      1. 2

        That sounds really cool! I’d consider buying a product like that for my workbench at home where I find myself regularly switching between my desktop, laptop, RasPi, and Novena.

    5. 5

      Work: Whatever I’m supposed to Agile to death, probably a lot of debugging, maybe a bit of refactoring. A lot of deep work, as I successfully implemented “hideaway time”, where I disappear and disconnect for a whole hour or more to focus. People have been receptive, I expect, mostly because of the immediate results.

      Home: I’ve made a few mods to my autoapi project to allow filtering of generated HTTP methods. I also make use of vendoring. We probably are going to use this in a support tool at work, maybe.

      Personal issues: We decided to cut off the landline and home internet, as it would save us a solid 80$ a month (along with Netflix), which is good. I realized I’m deeply addicted to my internet line when the idea was suggested, and I’m frankly not liking the idea. We do have cell phones with limited data plans, which is good, but this is pushing me way outside of my comfort zone and this semi-anonymous space seems like an appropriate place to vent my privileged north-american white male woes.

      1. 2

        We decided to cut off the landline and home internet

        The horror, the horror… j/k

        I can’t sever the home Internet connection but it’s been a little over four years with a VoIP account and cell phones. How are you managing without ‘net at home?

        1. 4

          I kind of involuntarily tried this for about 6 months, after starting grad school and having issues with getting internet hooked up to a new apartment. It wasn’t bad overall. I lived a few blocks from the university, though, where I could use the internet in either the library or my office. May have been more annoying if that weren’t the case. I ended up with a more regular sleep schedule, since I wouldn’t stay up all night online.

          1. 1

            Having three kids helped me get back to a regular sleep schedule. I learned to treasure it. XD

        2. 3

          Well I got an awesome connection at work, and then there’s a fair amounts of cafes, libraries, and other free access points around. My cell phones will fulfill the immediate communication needs, and the data plan on them will help with a punctual required access. I’ll just have to be careful about usage.

          I have a media server at home, with all my legally-owned movies and series, and I’m gonna host a few services “offline”. I wanna try to have a few wikimedia instances. Wikipedia and Wiktionary come to mind. Might have a “Ask” post about what services I should/could host offline. Definitely gonna have a godoc instance tool running at home.

          I’m really nervous about this; I think I’ve been feeding a web addiction from my early teens (first 56k connection when I was 10, never have been disconnected more than a week since, am now 29.)

          1. 3

            I’m really nervous about this; I think I’ve been feeding a web addiction from my early teens (first 56k connection when I was 10, never have been disconnected more than a week since, am now 29.)

            I don’t know you, but I won’t let that stop me from giving advice… regarding “Internet addiction”, it might be a problem if it:

            • interferes with other aspects of your life
            • displaces other things and activities that you value
            • distances you from people that you want to be close to
            • causes you anxiety when it’s absent
            • makes you think you have a problem

            Then I’d suggest changing your behavior, establishing new patterns, weaning yourself off. Otherwise, accept that it’s a time sink just like other activities (reading novels, watching movies, playing games, etc.) and specifically budget time for it.

            1. 1

              Mainly the two last points. First one too to an extent. I like to know there’s Internet where I’m going, it makes me anxious when i don’t have access, and i think that’s a problem. I don’t even use it that much at home anymore, other than a lot of Netflix, which I’ll substitute by a huge catalog that i will curate myself. Idk. Feels weird and uncomfortable. Definitely outside my comfort zone.

      2. 2

        But then how are you going to access porn?!

      3. 1


        1. 3

          Totally. I don’t even need the money that bad. Like i have savings. My total debt is under 20% of my yearly income. Canadian average is more like 150%. I’m super privileged. I realize this doesn’t make me sound any less of a dick but i do realize I’m in a good spot.

    6. 4

      Horn toot: In about an hour I’m launching the Concise Guide to Substitutability, a short ebook that explains what’s at work in several seemingly-unrelated topics in real-world Ruby code.

      This week I am attending MicroConf, a small conference for self-funded techie businesses. I’m here for the hallway track, to get advice from experienced devs about marketing and running a small businesses that’s starting with the above ebook and hopefully growing to be a full-time job helping devs write better code.

      1. 2

        @puschcx The book topic looks great, but I don’t write much Ruby. (Mostly JavaScript/TypeScript, though I can read Ruby.) How applicable would you say the book is to non-Ruby developers?

        1. 2

          It’s relevant to dynamic languages that make use of duck typing (JavaScript, Python, PHP off the top of my head). Almost all the example code is in Ruby, but I think only the mention of mixin modules like Enumerable is the only thing someone might not understand right away.

      2. 2

        Looks great! I’m a rails dev so right up my alley. Just purchased it.

      3. 2

        Congratulations on the launch!

    7. 4

      Daughter is out of school for term break this week and next, so we’re doing a bit of travel. Spending a couple of days in Bangkok now, then probably go to Amphawa for 1-3 days tomorrow, then back home. This weekend, we’ll head out to visit father-in-law near the border with Cambodia.

      For a change of pace, I was wondering about some aspects of Ted Nelson’s Xanadu. I thought that maybe a good way to understand some of it better would be to run some of their open sourced code. So … I imported udanax-green into git and started fiddling with it.

      I haven’t actually gotten it to work yet. The first issue was that sbrk doesn’t work on Mac OS X, but that was readily worked around. Now, there’s an issue with some of the file I/O code that results in lseek failing.

      I’d forgotten how bad really old C is. This is K&R C, so pre-ANSI … and it lacks anything resembling modern best practices for C at all. Now, I’m a bit less sure that this is a viable way to come to understand Xanadu better.

      I’d welcome any assistance with getting this running.

      Our maid decided to return home, so we’re looking to hire someone new.

      1. 1

        I’d forgotten how bad really old C is. This is K&R C, so pre-ANSI … and it lacks anything resembling modern best practices for C at all.

        That reminds me of using old versions of Microsoft C on DOS, pre-ANSI. And then later versions which were ANSI. IIRC the pre-ANSI MS C versions had a flag (/Zg ?) that could output prototypes for all functions defined in your code, to stdout, and then I used to add them at the top of the .c files. Workaround, but helped, if I remember right.

        1. 1

          The lack of prototypes is only one of the many issues. :)

          If I stick with it (unlikely), I can use cproto and clang-format to clean the formatting and things up a lot … but it would still require substantial changes to fix tons of warnings … much less deal with how it uses globals and all sorts of other issues.

          And my goal wasn’t to maintain this … I just want to understand parts of it better as I think some things apply to other work that I’m doing.

    8. 4

      Lots of ops. Using Terraform to hammer out a production deployment on AWS. Currently figuring out VPN for our VPC.

      It is a terrible idea to make me responsible for ops. Like to get back to Haskell code.

      Book’s final chapter proving to be a gnarly slog. Tired all the time. Just want it to be over now.

    9. 3

      Home My pet project has come a long way! It’s finally turning into a real usable program after several years of neglect. I’ve finally got the GUI to a point that I feel like it’s usable. And I’m starting to find and correct a lot of bugs thanks to the unit testing and just testing in general. I also managed to update the website to include a picture gallery. My apologies for my atrocious web design skills.

      Work Python programming as is usual. My title was recently changed from “Research Analyst” to “Intelligence Analyst”. Cracks me up because I’ve never earned either title in my time here.

    10. 3

      Trying to figure out what route to take in building up a new mail server on my freshly-minted Scaleway “bare metal” box.

      • Go through all the aggro of making exim/db/dspam/clamav/opendkim play nicely with each other, and luxuriate in the neatness of only running exactly what I want?
      • Or just go for iRedmail, have literally everything done for me, just put up with postfix/spamassassin etc and a more manual upgrade procedure, but save a whole bunch of time?
      • And while we’re at it, moving to a new box with new OS, should I just go along with “upgrading” to debian 8.2, just accept that systemd is here and get on with it, or should I stay with wheezy-backports and sysvinit and the concomitant gradual-death-by-a-thousand-deprecations over time?
      • Or, just ditch the whole lot, take the opportunity (and probably not-insignificant amount of time) to start learning OpenBSD and feel what looks like it might be a regular burn in dealing with an OS not frequently supported by VPS/etc providers?
      • And once there, dive into the whole opensmtpd+other-packaged-systems approach again, or just whack iRedmail on there?

      Thoughts/alternatives extremely welcome :-)

      1. 2

        A potential middle ground between 1 and 2 is taking what you want from something like sovereign.

        1. 1

          Interesting, having a look at that, thanks for the tip.

      2. 2

        I didn’t find the first route too complex. I use postfix, almost in the default configuration except for adding dkim on outgoing mail, and checking IPs against the Spamhaus RBL on incoming mail.

        1. 1

          Thanks, good to hear.

          I think I’d be happy to take postfix on if it was part of a managed package with some community support like iRedmail. If I end up doing my own build I’ll probably go for exim, as I know it somewhat, and like it, vs postfix which I don’t really know at all. Which ignorance is largely because Wietse Venema & djb’s spat over security & trustworthiness still leaves me residually reticent about postfix, ‘cos I worked a fair bit with qmail back in the '90s, really rated it, and overall preferred pretty much everything I read by djb over pretty much anything I read by WV. (I also like most things I read by Philip Hazel, though it seems he doesn’t work on exim any more.) Maybe I should just get over it, it was all a long time ago … right? ;-)

          1. 1

            There’s probably some gray area, but I tend to divide mailserver setups into more “unix-style” setups on the one hand, with pieces installed from a distribution (Debian packages, BSD ports, etc.), and configured/managed in a traditional style, and then more “app” style approaches (iRedMail, Zimbra, etc.), typically installed as a self-contained third-party binary distribution and with one-stop configuration. I personally prefer the first style, but I do know people who prefer the second.

            As for postfix vs. exim, I don’t honestly have a strong reason to pick one or the other. I liked postfix’s documentation, but exim would probably have also been fine.

            1. 1

              Yeah I know what you mean. Overall I prefer the first style too. I’m just getting to the stage where I just want it done, and as a lot of smart people have packaged things up really nicely the temptation to just use the packages is strong. In that vein I tried Zimbra a while back, but it was super-heavy-duty, kind of determined to take over everything, and I didn’t like it much. Thing I like about iRedMail is that it bundles together all the necessary packages and installs them using apt-get so it’s not just an opaque binary, and seems to hybridize between 1 & 2 somewhat.

      3. 2

        OpenSMTPD works on Linux too!

        I’ve heard rspamd is the new hotness for filtering, but I’ve not used it myself.

      4. 1

        How is Scaleway? I’m thinking about transitioning to them from Digital Ocean soon; their bare metal ARM machines seem fun!

        1. 2

          Yep I’m liking it so far. The tools are decent, the UI is fine, and the features are enough for me - basically VMs/snapshots/custom-images etc, S3-compatible storage, and a decent-looking API. Obviously not as full-featured or having a 10th as many different products as GCE or AWS, but it seems a really good core offering. It has a good bunch of ready-made images, decent OS/distro choice (though not OpenBSD like I said), and seems solid. Network/connectivity/bandwidth/allowance seem great, though I’ve yet to do anything significant intercontinentally - it’s in Paris so it’s great for me in London. Prices are great. The issue right now is capacity - you can get the x86 VPSs or the ARM instances immediately (though the ARM says “limited stocks, getting deprecated soon”), but for the bigger x86 “bare metal” machines, it took over 24 hours for me to get even a small one up and running, with an opaque “unable to allocate” error message coming up. Support got back fairly quickly and said “yeah, we’re building capacity right now, but for the moment, you have to wait for someone to switch theirs off before you can have it”. Which … O_o. But I’m taking it as a sign that, yeah, there’s a lot of demand ‘cos it’s good. We’ll see. I read a few people saying its owner Iliad has some dodgy transit policies but I’ve yet to dig much into that. Overall: seems good :-)

    11. 3

      Last week I got pretty side tracked with the farm. I attended a conference on sustainable living, and sat in on a couple of talks about low maintenance sheep raising systems, which doubled my desire to raise a small flock of my own.

      This week, I’m planning to finish those data structures. I’ve already wasted enough time, and I shouldn’t have too many distractions outside work/side work/animals, this week.

      1. 4

        This is the most intriguing comment I’ve seen on this thread. Farming and data structures seems like a rare mix of activities. I’d be interested to know more of how that came about.

        1. 2

          I’m not sure that this thread is the best place to talk about it. If someone’s interested, shoot me a message and I’ll get back to you.

      2. 2

        Doing anything like permaculture? Saw some of that and an earthship recently.

        1. 2

          I’ve been keeping myself pretty busy with other projects, but one of the things on my list is to get cattails growing near where some of our drain water runs off to act as a filter before it collects in a small basin we’ve dug out. Hopefully it’ll cut down on how much we have to actively water all the animals.

    12. 3

      Work: More dissertation writing… I’ve finally found the time to transition from sublime to emacs which has been fantastic so far. The learning curve for emacs is far less steep than that of vim I think.

      Home: I spent the past few days on a family holiday which was very much needed for me in terms of avoiding burnout. I also managed to collate the family photos into one place and am backing them up on tarsnap.

    13. 2

      I’m prepping a talk on Ruby Object Mapper for my local ruby user group. Struggling with trying to find non-contrived examples of active record vs CRQS currently.

    14. 2

      Trying out FreeBSD for a new personal webserver I’m setting up, and trying to remember how the hell you configure wordpress to be slightly less insecure & letting PHP write everywhere. Mostly WP seems to be going “Gimmie your FTP details!” which is slightly … concerning to say the least.

      1. 1

        I know it’s not a perfect solution, but wouldn’t running WP in a jail and proxying it from the main web server isolate the harm it can do?

        1. 1

          I think I mostly need to just make a decision and get it setup. Can always change it in future, and it won’t be perfect no matter which approach I take. Currently there’s only one site on this server (which is just a VM), so there’s not actually that much at stake if WP did get popped. Jails are a good idea once I put a second site on this VM though, thanks for the reminder.

    15. 2

      Finally back home in Buenos Aires! Luckily my family took care of me having a decent place to live when I arrived, so I don’t have to do much (buy a table, a couch, maybe a TV).

      I have ~15 interviews lined up for this week. I’ll go to all of them and then decide which place I like the most, maybe I’ll start during next week. Deciding work affects your bank account and health coverage in this country, so being if I was going to be unemployed for a month I might need to get those myself and then change, which is kind of messy. Also it’s Monday 9am and I’m already bored of being home, I feel like I want to do something, not burned out. I enjoy routine.

      The only problem about my relocation was that my cats were not able to flight because of some airline fuck up, so they’ve stayed in San Francisco with a friend and I have to figure out how to transport them. In the meanwhile, I’m missing them, which is quite sad.

      1. 1

        Wow, 15 interviews?!?! I usually feel lucky if I can get one in a month. Is the market there desperate for people with your skill set?

        1. 5

          Apparently… With the currency exchange rate, many companies outsource to Buenos Aires and pay very little for them, which translate into a lot here.

          Most people do not get this many, but I’m coming back from working three years in the San Francisco Bay Area, and some of that time was for Twitter Inc, which was good experience and good for my résumé.

          I’m taking a huge pay cut in absolute terms (I’d be really lucky if I earn half of what I was making) but it’d be more than enough to live comfortably.

    16. 2

      A little application in C#. I’ve split the backend into a portable library, and so now I have a Windows and GTK frontend. Adding fit and finish, as well as mobile and Mac UIs are something I want to do.

    17. 2

      This week, at work, we are shipping our new, kafka based, metrics pipeline which will open the door for lots of new things in the next 6mos, which is amazing.

      At home, I need to do some work on cfdg, and I would like to reach out to a few people for a potentially new hifi by apg, which I haven’t published in months.

    18. 2

      Work: Bringing our teetering Hadoop cluster back from the brink of disaster. This is exactly as much fun as it sounds.

      Personal: Rust all the things! I ported an incremental backup script originally written in Python to Rust a month or so ago and finally got it rolled out (and it’s working great), and this weekend I threw together a self-instructive OpenGL demo. I would love reviews or recommendations on both of those!

      1. 1

        What causes a Hadoop cluster to teeter?

        1. 3

          In our case, it’s insufficient capacity combined with a severe lack of proactive maintenance and administration. We’ve failed to keep up with software updates, so a combination of bugs and hardware failure have resulted in insufficient data replication (no replication for some data, which is scary), uneven data distribution, and rebalancing failures.

          In general, the software is pretty fragile. It’s incredibly complicated and there are a lot of critically important code paths that almost never get exercised. I’m fond of this blog post for reasons not to use Hadoop, but high administrative overhead is at least as strong a reason.

    19. 2

      Working on an active social app in Rails 5 to get up to speed on new features/ActionCable.

      Applying for as many junior development jobs as possible.

      Doubting my worthiness.

    20. 2

      Prepping for migration of a Postgres DB to RDS. Started an #emacs-curious channel on our work slack team, and have fielded some questions there from 3-4 colleagues more used to Vim.

      Learning to use Emacs/gnus for email, writing harmless mail in (you guessed it) org mode.

      1. 1

        “Harmless”? I meant HTML here. Silly autocorrect :-)

        HTML mail is good when tables are involved, since few people read email with fixed-width fonts any more.

    21. 2

      Still thrashing away on NeuralObjects, our new Machine Learning as a Service offering. Right now I’m focusing on the user signup flow and about to start on Stripe integration so we can, like, collect money. :-) The basic workflow of:

      1. upload data via WebHDFS

      2. Submit a job via REST API

      3. collect results via WebHDFS

      is working,although there’s still work to be done on the backend API service. We also plan to add support for a prediction API so that once you’ve built a model, you can make predictions directly via API. And, of course, we’ll be adding the ability to upload your own custom code both for model building and for predictions.

      We’ll also be exploring whether or not there’s a need to implement something like Tsunami and/or GridFTP for uploading large datasets.

    22. 2
      • Making good use of my employer’s new continuous delivery pipeline to get bug fixes deployed without a lot of hoopla.
      • Putting the finishing touches on a fantasy novel I’m self-publishing. (Formatting, author bio, cover, etc.)
      • Starting to pack for a cross-country move next month.
    23. [Comment removed by author]

    24. 2

      Webhook/JSON transformer, extractor, filter, proxy running on lambda (serverless).

    25. 2


      Term project (web architecture) and finals (cognitive science, computer vision).


      Figure out what’s causing my laptop to freeze once in a while. Not sure if it’s my window manager (awesome wm) or X11, or kernel problem yet (arch linux). Happened since last system upgrade, so need to check my logs.

      I also just bought one of those cheap 3D printers lately. This is the one I bought. Heard good things about it and it’s the only one I’ve ever found at this price point that’s not made of acrylic (it’s aluminum!). So i’m going to have a great time building this and dialing down the settings!

      This might also be a good excuse to get another raspberry pi (rpi model 3) to run Octoprint as well ;)

    26. 2

      Personal: still cranking away on Paperdoll (a scala implementation of “Freer Monads, More Extensible Effects”). I’ve had a bit of scope creep in what I now consider required for 1.0, but I feel like I now have a much clearer sense of where the project lies on the idealism/pragmatism spectrum and I know what the remaining pieces are and how to do them. Now it’s just a case of cranking the rest of it out.

      Bizarrely enough hardware is starting to be a limiting factor: my laptop has a dodgy fan and overheats and shuts down when I build the project. Can anyone point me to an ultrabook (i.e. thin and light, ideally 13" or smaller) available in the UK with NVidia graphics, 12gb or more of RAM, reasonable processor, and touchscreen (bonus points for 3D)? Currently I have a portable machine and a gaming/development machine and I’d really like my next one to be able to replace both.

      1. 1

        For the record I went with the Microsoft Surface Book. Not the manufacturer I was expecting to go with, but it’s a beautiful object and I’ve been happy with it so far.

        1. 2

          I hear a LOT of really good things about the Surface. I’m pretty heavily invested in the IOS ecosystem at this point, but if I ever actually need a super portable laptop and IOS can’t cut it for some reason, I’d definitely give Surface a look.

          1. 1

            I’m not treating it as an iOS-like device; for me it’s a full x64 workstation that happens to have a detachable screen as a bonus. I wouldn’t even call it super-portable (it’s 13-14", it has physical home/end/insert/pgup/pgdn keys (though it does the “fn key” thing and doesn’t have a number pad)). My own ecosystem investment is in windows games, including some older ones that will never be ported to other systems (e.g. my LAN party group still plays Total Annihilation).

    27. 2

      In between jobs, I need to do tons of paperwork, and refresh flask, sqlalchemy and angular. But I just started a new client side form validation library in js. Damn cross validation

    28. 2

      Making an installer for linux-based distros (debian for now, but not making it debian-specific from the start) completely from scratch for the needs of the company I work for. Will probably take more than a week.

    29. 2

      Trying desperately to finish incorporating revisions from my collaborators on my latest paper. Morale is low, distractions abound.

    30. 2

      I finished the SANE support for GTS (Studio Ghibli’s recently open-sourced scanning tool): https://github.com/stefantalpalaru/GTS

      Right now it simply uses the first available SANE device, so I need to (dynamically) add a menu item to the FLTK interface and allow scanner selection. That, and somehow show the selection window when the scanning back-end is initialized with no device selected, before the main GUI loop is entered.

      I could use a OS X tester, if anyone is interested.

      While I do appreciate the architectural decisions that made the porting straightforward, the code quality and the interface’s usability are subpar. I wonder how many internal tools are like that, and why aren’t the people in charge tapping into the mostly free resources of the open-source community a lot sooner. It’s not like scanning / post-processing software like this one is what gives an animation studio its edge over the competition.

      I’m conflicted about making more significant changes - is it more important to preserve a piece of history, or to improve the tool?

    31. 1

      I just found employment in the nerd world and now I participate as a professional rather than as a hobbyist and onlooker. My company is building an online learning tool and the idea is that I eventually become the lead developer – not as a matter of experience but because it was built in the spare time of two developers who are busy doing other important work.

      I’m enjoying myself. (=

    32. 1

      Third in a series of command line utilities in Python - a binary file split utility:


      The previous two - a (text) file split utility and a file compare utility - are linked to from the above post.