1. 28

This is the weekly thread to discuss what you’ve done recently and are working on this week.

Please be descriptive and don’t hesitate to ask for help, advice or other guidance.


  2. 20

    It has been a long while since I’ve posted in these. I had been struggling with a major depression lasting over 2 years, but finally broke free in May. There were some really difficult times along the way when I wasn’t sure that I’d survive the week, but I did.

    We’ve been doing a lot of travel this summer with trips to Bali and other places as well as hosting various visitors for their tropical holidays, but our daughter is back to school next week, so things get to calm down.

    For my client work, I’m still working with emscripten.

    For my own work, I’ve been letting myself flit from thing to thing like a butterfly in a field of wildflowers. I’m working on some fixes to LLDB. I switched the Open Dylan documentation over to using a theme based on the Read the Docs theme (example here).

    This week, I’m working on improving some aspects of the Windows build for Open Dylan and might get to improving Windows support. (I recently added core support for outputting colored text which needs to be implemented for the Windows console for example.) I’m also working on fixing some issues that stand in the way of integrating the memory pool system GC with our LLVM back-end. (It has been used extensively with one of other compiler back-ends since inception over 17 years ago.) I’ve also been working at fixing an issue in our LLVM back-end that breaks my Objective C bridge.

    Finally, I’ve recently put some money forward for the ProseMirror fundraiser. This looks like excellent work by the same person that does CodeMirror. I’m really hoping that this gets funded as I think it will be useful for many folks and Marijn deserves our support.

    1. 3

      Major congratulations on breaking free. I was drunk for most of the last decade and because of that was not able to recognize serious depression. Now, sober, my “what are you doing this week” (and every week for the last couple of months) is trying to build healthy habits around techniques to manage it.

      1. 3

        Congratulations on surviving that. I’ve been there, I know it’s not easy.

      2. 7

        Last-minute studying for the Ph.D. prelim exams. They’re on Wednesday. Hope I pass.

        As far as research goes, still working on adding AXI4 support to Rocket Chip. This should allow people to start building interesting RISC-V SoCs.

        1. 2

          Best of luck!

          1. 1

            This is super awesome!

          2. 7

            Working on adding various offline analyses of heap graphs for Firefox and SpiderMonkey. Finishing up the “census” analysis which accumulates and counts every node in the heap graph based on user-specified buckets such as “constructor name” or “allocation stack”. Next up is computing dominator trees so we can calculate retained size and say “this object X is preventing that object Y from being reclaimed by the collector” aka “all paths from the GC roots to object Y pass through object X”.

            1. 6

              For money, I’ve started my week in python, will probably continue it in php, and end it in agile ceremonies. I’ll probably try to plan a lunch and learn in the office, about Go. I did an introductory one last week, overwhelming success I believe. I’m thinking maybe something in connection to Lambda architectures and/or DDD would probably excite a few people around here, so IDK. I wanna be doing more of that, so I get better in front of bigger crowds.

              On a personal level, I’m slowly coming back to what I think may have been a close call with a burnout, and I’m starting to find the will to work on my personal projects again. I think I could do clever stuff to add to the magic of Autoapi. I also found out there’s a php thing called autoapi. I might have to find a better name for autoapi.

              1. 1

                Hm, isn’t that exhausting? Python, PHP and Go in one week? :-)

                1. 1

                  I find PHP to be the most exhausting of them all. Apart from that, I don’t really mind the language-switching.

              2. 6

                Just re-enrolled for college. I’ve been working full time for a year now at a VoIP company and put four months into a startup that failed to launch after the other co-founder quit (I worked on the startup part time). Pretty excited, I think my life will be a bit easier once I get a degree. I’ve had interviews with good companies but my lack of formal knowledge always leads to me being embarrassed and second guessing myself in the interviews.

                Other than that I’m trying to find vulnerabilities in my friend’s smart TV.

                1. 6

                  My final year of university started today!

                  I’m taking:

                  • Analysis of Algorithms
                  • Intro to Cryptography
                  • Intermediate Russian I
                  • Probability and Statistics
                  • Concepts of Parallel and Distributed Systems

                  I’m really excited for all of these classes – there’s going to be a lot to learn! I’m going to look into potentially working on the programming projects in multiple languages just to get my feet wet. We’ll see how things go!

                  1. 4

                    At work, been putting a lot of time into a query profiler for Elasticsearch. When profiling is enabled, you’ll now see a timing breakdown for each “node” in your query tree at each shard involved in the search. You’ll also see a tree of Collectors (Lucene objects which “collect” documents to score) which can provide an overlapping set of timings about how query vs post_filter vs aggs vs global aggs interact.

                    The response is super verbose and a bit quirky due to Lucene mechanics (e.g. queries can “rewrite” differently depending on local data, so different shards may have slightly different representations of the query tree) … but I think it’ll be a huge boon to anyone trying to debug queries. In the past profiling queries relied on intimate knowledge of Lucene / ES and divining tea leaves, which is obviously not ideal :)

                    For fun, I recently finished a MYOG ultralight backpack. Weights 7.5oz, ~40 liter capacity, water resistant. It’s survived a few 46er hikes so far, so I’m pretty pleased. Starting on the next version, which will be a little more rugged and a lot less voluminous. Also getting back into Rust + Mio side projects.

                    1. 2

                      I would love to hear more about both of these things! Any chance the profiler will be FOSS? Got a blog post about either one up your sleeve?

                      1. 1

                        Whoops sorry, I probably should have stated that I work at Elastic :) If all goes well, the profiler should make it into ES 2.1 core. It’s still pretty rough (I just pushed the shared branch, we’ll start iterating on it soon), but you can follow along here: https://github.com/elastic/elasticsearch/tree/query_profiler

                        There is a bit of history and explanation in this PR too: https://github.com/elastic/elasticsearch/pull/12974

                        I’ll definitely be writing an article about it, once it get’s merged. The output is a little tricky to read, so I we’ll need a primer on what the various components mean, and how to interpret the results. Keep an eye out for it sometime post 2.0 :)

                        My personal blog burned to the ground a while back, so nothing written about the backpack yet. There are some photos here though: http://imgur.com/a/mkCfZ

                        1. 1

                          Looks like the profiler will be pretty handy. Your elasticsearch-inquisitor looks nice, too. I’ll have to have a play with it next time I’m debugging our search results.

                          Is the pack based off a pattern? I’m a little hesitant about UL packs because I like structure you get with an internal frame.

                          1. 1

                            Fair warning, Inquisitor has been essentially unmaintained for a long time now…I’m not sure if it works with modern ES. It should theoretically work, since it is 1.0+ compatible, but you may find oddities or bugs :)

                            No real pattern for the pack, just cobbled together from various articles on the internet and studying backpacks that I own. The main problem with my pack is that it’s too large: it can technically hold ~40 liters, but the lack of structure would make it awful (and probably not strong enough without ripping either). Realistically it only holds ~15lbs, which works fine for day trips….but I wouldnt want to use it for an overnight or longer.

                            My next pack will be smaller and more rugged, to help maintain it’s structure more and less likely to overload. Some day, I’d like to tackle an arched external frame, where two carbon fiber rods are tensioned to provide structure and lift the bag off your back. Need a lot more practice first :)

                    2. 3

                      I’m going to try backtesting some trading strategies using Pandas and see if we can get some deployed, although maybe this is not the best week to do that.

                      In my spare time, I’m hoping to get lists.canonical.org back online (as a web server at least), and I want to finish writing an update to “choosing powerful primitives for a simplified computing system” that includes a bunch of new stuff I’ve been learning about.

                      Also, I’ve been thinking that maybe if the Stepanov/coqasm approach is the right one, then maybe instead of Stoneknifeforth, the ideal minimal bootstrap compiler might be a simple term-rewriting system, like Aardappel, being used as a retargetable macro assembler, also inspired by this HN comment by jhallenworld about generic assemblers.

                      Unfortunately when I hacked together a simple term-rewriting system the other night and used it to do some simple symbolic differentiation, it was like an entire page of Scheme! (Plus 13 lines of symbolic differentiation rules.) I hoped it would be simpler. It’s possible I’ve overcomplexified the problem of term-rewriting, but I feel like maybe if it’s an entire page of Scheme then it will be probably several pages of assembly language, and so maybe term-rewriting isn’t the best bootstrap to start with after all. StoneKnifeForth is only about a page of code, and in theory can compile itself, although I need to figure out why the ELF it generates is rejected by current Linux kernels. But maybe a term-rewriting macro-assembler is the thing to build on top of StoneKnifeForth as a stepping stone to higher-level programming, rather than something like Ur-Scheme.

                      Recently? Well, this is Karplus-Strong string synthesis in obfuscated C:

                      /* Karplus-Strong string synthesis (“Digitar”) in two lines of C. */

                      If you’re on a platform other than Linux, you’ll probably want to change aplay to whatever command will play an 8000-sample-per-second mono 8-bit linear unsigned stream of samples from stdin.

                      I think I’m making some real progress on putting APL and similar array languages (Numpy, R, J, K, Pandas) on a simpler informal axiomatic basis. I wrote a very limited exploration of these ideas last weekend, but it isn’t very runnable yet.

                      And I think jumprope-style, bup-style, or ssdeep-style hashsplitting can be made somewhat more efficient (by picking blocks of more consistent sizes) and dramatically more robust against maliciously chosen data, at a modest cost to the consistency of where it breaks blocks, by dynamically adjusting the hash rareness threshold down by 1 every byte and up to 8192+current_rareness when we hit a hash that beats the hash rareness threshold and thus start a new chunk. The tradeoff of size consistency vs. consistency of split points under insertions and deletions is determined by the distribution of rarenesses: if your measure of rareness is simply the number of trailing zero bits in the hash, you will get more consistent sizes and less consistent split points than if it is, say, two or three times that number.

                      1. 2

                        Just got Myrddin (http://eigenstate.org/myrddin) to be largely ABI compatible with C. (Passing and returning structs is still broken). Now I’m doing a bit of work on more performance – I’d like to get pattern matching much faster. I’m also poking at the parser generator (http://git.eigenstate.org/ori/mpgen.git)

                        Also hacking on productionizing FBOSS (https://github.com/facebook/fboss)

                        1. 2

                          Last week I was busy trying out NixOS on my company issued laptop, which was amazing! I gave up on it when suspend didn’t just work out of the box and installed Xubuntu (I’ve never liked ubuntu, but figure it aligns with other Linux users in the company and will likely be the basis for a company image). Might try NixOS on my own machine some day again.

                          My implementation of cfdg in Racket is usable now, and though I have additional ideas and work that I wanna do with it, I’ll likely put it aside.

                          Yesterday, I threw out a ripped up copy of the Star Wars Scanimation book, and thought it might be neat to make a program that takes a GIF and produces a scanimation image. The whole thing is super neat! If I do this, I’ll certainly write a blog post about it.

                          1. 1

                            Do you think you’re going to use Nix or Guix on Xubuntu, without installing NixOS or GuixSD?

                            1. 1

                              It’s possible, but I feel as though to do it right, I’d need to strip Xubuntu down to the bare minimum, which might be annoying. Maybe I’m wrong though? What are the best practices for using nix-pkgs on top of another OS?

                              1. 1

                                I don’t have enough experience to answer, but one of the other things I’m planning on doing this week is installing Nix on top of Debian in order to get a modern version of Pandas.

                          2. 2

                            For the startup: Working on fixing Quoddy bugs1. Specifically, I’m rewriting the way we query for items to render in the main “stream” view to make it a bit cleaner and fix a couple of different - but related - bugs.


                            I’ve started making a real pointed effort to add some new languages, libraries, and tools to my repertoire. See http://mindcrime.github.io for more details. I have a laundry list of things that I’m at least dabbling in, but the ones I’m really focusing on for right now include R, Common Lisp, and ACT-R2. I’m also making a real effort to get up to speed on using React on the front-end.

                            Re: Common Lisp - I’ve been an Emacs user for years, but never an Emacs power user. But deciding to focus on Lisp finally got me motivated to get Slime installed and has forced me to start learning my way around Emacs a bit better. I still have a lot to learn, but at least I can edit a .lisp file, compile and run it from within Emacs using SLIME, switch to the REPL frame, run more code, switch back to the frame with my code buffer, etc. I feel like I’m finally joining the civilized world. :-)

                            On a semi-related note, due to my overall interest in AI and ACT-R, I’m also trying to squeeze in some time to learn more about OPS53, an older “production rule system”4 environment, because I want to understand more about how these things work.

                            1. 2

                              At work, I have some ops stuff to deal with. Mostly coordinating salt changes with other people.

                              Outside of work, making a concerted effort to be more effective at Common Lisp so I can use it more. Today’s stumbling block was not being able to figure out how to access Unix domain sockets. Fortunately such stumbling blocks are pretty rare.

                              I’ve also got a weird itch to write my own editor, which means doing a lot of research ahead of time and knowing what exactly I want out of my editor.

                              Still struggling with a smouldering burning out.

                              1. 2
                                • Dropping IE9 & IE10 support. (Woo! Flexbox!)
                                • Continuing to build an internal CSS framework at work
                                • Writing another post for my company’s engineering blog
                                • Beginning the next chapter of my steampunk novel
                                1. 1

                                  Beginning the next chapter of my steampunk novel

                                  How many chapters do you have so far, and what is it about?

                                  1. 3

                                    Thanks for asking. :-)

                                    Four chapters so far. The main character is something of a cross between Batman and Joan of Arc, and she’s trying to stop a serial killer during the Chicago World’s Fair.

                                2. 1

                                  Automating Openstack infrastructure with Ansible.

                                  1. 1

                                    YET ANOTHER rewrite of one of our apps, but then never again.

                                    Finishing a cute trigger to calculate page visit duration from our analytics stream. The more I work on that, the more I’m convinced that nobody actually needs Google Analytics for basic traffic stuff. It’s a fun exercise.

                                    Buying power supply parts for a BeagleBone Black and RPi2 cluster, and bringing up a refurb server I literally bought out of the back of a car the other night. :)

                                    Oh, and seriously considering getting a roomba to fend off the massive cottonball invasion my dog is summoning every six hours in my house. Any recommendations?

                                    1. 1

                                      Got my app capturing arbitrary frames from a canned video; now, integrating a loop to caffe for classification, live video support, and pretty power usage graphs. I also have a huge pile of PC parts sitting on my office floor waiting for assembly; I managed to order two 5930ks and zero power supplies, so it’s all just inert until Purolator delivers the PSU (and second Titan X).

                                      In non-work stuff, I am steeling myself for a second go-round at getting better at org-mode; recording some songs with the band; and attending to the needs of a tiny tyrant.

                                      1. 1

                                        The final rollout phase of our SAP/HTML5 mushroom farm logistics project is rolling out next week: once that’s done I’ve got to write up some kind of highlights & lessons learned blog post. I’ll be presenting at OSDC 2015 in Hobart about it too. After that I’ve got the final merger of a customer portal project I developed 8 years ago into the acquiring company’s customer portal, sad to see it go but on the other hand it has had a good long life!

                                        Then I’ll be looking for the Next Big Thing :-)

                                        1. 1

                                          @work I’m fixing bugs in our Celery code and updating a Django project from 1.6 to 1.8. I’ve hit a few snags with the new migrations framework causing problems with foreign keys which is going to be a real pain to figure out. I’m also trying to kick off a regular lunch ‘n learn.

                                          @other I started getting up at 0530 again to get to BJJ class before work. It’s not easy getting out the door so early in the morning but I always feel less stressed at work when I hit the mats first thing in the day.

                                          1. 1

                                            Recovering from being sick as a dog last week. Laid up in bed for several days. Very nasty.
                                            Hoping to start feeling like a person again this week!

                                            1. 1

                                              Last week I sorted out the correctness and speed issues in our JS erasure codes implementation for Peergos and spun it out, many thanks to @sirpengi for advice and suggestions. Interestingly Firefox is much better than Chrome at optimising the erasure JS. The worst case with 10 errors takes 14s in Firefox compared to 80 seconds in Chrome.

                                              Meanwhile we’ve got the back end side of follow requests (without exposing the follower) working. The demo is coming along. Next up is getting all the erasure code stuff working in web workers.

                                              1. 1

                                                Working through the project Euler problems in Racket as a way to learn functional programming. My adventures are at https://mathandcode.wordpress.com/ and the github repo is at https://github.com/kghose/euler

                                                1. 1

                                                  No work on Fire★ this week. I decided to refresh up on my algebra and just ordered George Chrystal’s Algebra book.

                                                  I am beginning to think math is the most important thing a programmer needs to know and algebra is a must. I heard George Chrytal’s work is one of the best.