1. 23

This is the weekly thread to discuss what you’ve done recently and are working on this week.

Please be descriptive and don’t hesitate to ask for help, advice or other guidance.


  2. 11

    I thought this was going to be a smaller week … but ends up it wasn’t, I guess.

    I’m in Jakarta and having some nice down time, while not blowing my Github streak. Also getting some batik shopping in…

    I’m debating between upstreaming some code that I have for providing JavaScript bindings to the LLDB public API vs ditching that and writing a C API wrapping the C++ API which is much easier to bind. (JS is tricky, not just due to all of the various JS engines, but V8 changes API frequently … and Node.JS 0.10 vs 0.12 vs io.js … it is just a pain.) I might be willing to hire someone to write the C API binding for the C++ API (but at a somewhat low fixed fee).

    I started a semi-private mailing list last week as I want to conduct an experiment. There are a number of topics that I’m interested in (some of them going back 20 years almost now) and it is hard to find many places where they can be discussed in a way that isn’t specific to some subset of topics). So I’ve started a list and invited some initial people to it and I want to have discussions on things like

    • Object capability systems
    • Models and implementation techniques for containment, confinement and isolation
    • Systems where there are multiple users providing code who don’t trust one another
    • Thinking about the boundaries of trust within a system, especially one which is meant to be programmable by multiple people who may not trust each other
    • What happened to standardization efforts for many common technologies in use today (since many things are now siloed off into various competing commercial entities and companies want to own their users)
    • Other things related to the above, sometimes tangentially

    I’m hoping to have a list where a decent subset of the users actually post and share in learning … I’m thinking about having a blog where members of the list can guest-post with summarizations of various topics with varying levels of detail. It’ll be interesting to see how it goes.

    Meanwhile, in Dylan land … I’m working on some compiler bugs and other miscellaneous things. I also did some code review on a large set of changes to start implementing the language run-time for the LLVM compiler back-end.

    And for work, I’m looking at the size of emscripten’s generated code and thinking about some possible improvements to make it smaller. I also added support to the emscripten tracing library (that I also wrote) for integrating with the Google Web Tracing Framework.

    1. 8

      The mailing list idea is great. Since the dawn of social (aka Eternal September #2), it feels like there are fewer places to deeply discuss programming. Link aggregators certainly aren’t it, and Twitter has anti-intellectualism as a core feature.

      Forums have always been great, but I don’t know of many. I think I just miss Usenet.

    2. 9

      Added that index I was talking about to the book using sharelatex’s documentation, making it easier to keep track of when we talk about or introduce topics. Not using a concordance because the verbosity would be unwelcome. We just want a sense of the first few times something is mentioned.

      I did a projection of how many pages it would take to finish everything listed in the ToC based on what we have done so far and the calculation was biblical. There will be blood.

      Trying to decide what I can cut since the current ToC already had a bunch of “nice-to-haves” cut out. The hope I’m operating on is that the basics require more explanation and demonstration and that later chapters on the book can be more breezy. The good news is, this will almost certainly not be my last book, so anything that gets cut will still get written for something else.

      I want a Trifecta parser for Haskell for supporting tooling so I’ve been poking around parsers while drinking my tea/coffee. Trifecta is a big part of why the Idris tools have such pretty errors & colourisation. That and putting the work in :)

      I’ve started remedying my woeful math education, in the process I’m collaborating with somebody on accumulating the tribal knowledge around mathematics into an open resource. We want information like, “where has group theory found application?” and “What are the connections between topology and homotopy type theory?”

      Giving serious thought to implementing MonoHaskell, a parody that made some people really angry.

      Eye-balling webdriver as a possible solution to some scraping problems a friend is having.

      A possible Haskell contract coming up as well.

      I am not in Jakarta. Or Yalta.

      My dog turned his head away from my face before sneezing for the first time today.

      I bought a nice condenser microphone for the Haskell meetup this Thursday in Austin, Texas.

      1. 5

        $school - next round of programming projects just got released so my free time is gonna be curtailed until I get that stuff out of the way again. More C hacking approaches…

        Oxlang - Stalled since I started reading into Rust and Haskell again. After extended discussions with @cemeric and others disaffected with the state of Clojure as a language I think the verdict is that what I’m personally looking for is a garbage collected imperative language with a combination of mutable and immutable datastructures, a more expressive type system and pattern matching. Turns out that Rust fits that bill quite nicely without my running around making another compiler so this one is back burnered.

        Grimoire - Finding some traction again as of last night. I’m working on hardening the lib-grimoire interface by reworking it atop a new tagged values macro inspired by @jneen_’s talk at Clojure/Conj and using tagged value contracts and matching pervasively. I’m also finally adding test coverage to lib-grimoire which is hovering near the 65% mark after last night. Once I get the library itself hardened to my satisfaction then the Grimoire webservice will receive a matching treatment (Est. 2-3 total afternoons of hacking). The goal of all of this work is to put Grimoire internally in a better place so that in the near future I can add example submission and editing to the website. Also in the works but farther down the pipe is a full UI overhaul and work on lein-grim, the documentation structure emitter so that ClojureScript is correctly supported.

        Detritus - Co-evolving with lib-grimoire as mentioned above. I feel like there is some major code movement between the two projects impending because some of lib-grimoire is really utility code that could be elsewhere and like all the Clojure utility libraries detritus feels entirely too big/monolithic.

        Back to class… arrdem

        1. 5

          So last week I said I was trying to get me head around promises…. in C.

          So I have done that now.

          I have an implementation of promises in C suitable for an embedded RTOS message passing environment.

          One cute feature is synchronization is done via lock free C11 __atomic’s.

          Now…. to convince the rest of the team that this solves an interesting set of our problems…

          1. 3

            Show us the code so that we can hold an ABA-searching race! :-)

            1. 1

              Good idea, I’ll just have to do the wrangling to get permission to open source it, that shouldn’t be too hard.

              Fortunately I suspect this case is less prone to ABA races as promises move from state A (pending) to state B (resolved)…. and then, sort of the whole point of them, never move back…

              However, I can easily imagine misuses of the API that would be an ABA race….

              1. 1

                Sigh! Now back to the Yak Shaving of bumping our target gcc to 4.9.2 (the current version we have 4.2.3 doesn’t have C11 atomics, something similar, but irritatingly not quite).

                ps: The concept of Yak Shaving is almost as valuable as the concept bike shedding .

                Those concepts and being able to recognize them in the wild can save a huge amount of time.

          2. 5

            Despite not much hacking time, I got a lot done on Hython:

            • Splat operator in function definitions and calls
            • Augmented assignment operator
            • Beginnings of the for statement

            I’ll need to expand out support for for statements more to include iteration, and iterator classes, as well as supporting break and continue. I’d also like to convert the lexer to Parsec, and finish off the parser. We’ll see what I get to this week.

            Also, I’m close to 300 commits on this. Crazy.

            1. 4

              $work: There are some weeks where you wonder if there really is a god out there, because if there is, I’m pretty sure it’s mad at me. My increasingly Job-like job (that joke won’t make sense to a lot of people) has reached a new peak of hilarity. The guy with all the keys to the candy shop, deployer-in-chief, and proud owner of a bus-number less-than-or-equal-to one up and got hit by a (metaphorical, resignation shaped) bus. Now we’re in a mad dash to figure out how to get me, my boss, and a coworker approved to do deploys so we hopefully can avoid weeks like this in the future. Already I’ve put in quite a few after hours hours, and a bit of work over the weekend. I suspect there will be more of that in the near future as I absorb and attempt to comprehend this particular slice of Ryleh our deployment.

              !$work: Missus is going to Austin for QuiltCon on Wednesday. She’s a fabulous quilter, and she’s quite excited about going down to meet up with some of her friends and learn more about quilting from all the famous folks down there. What that means for me is that I will have the house to myself for a fairly long weekend, that usually means recording, but between the $work stuff above and a recent bout of RSI, I’m not sure how much I’ll get done. Will probably just relax and do nothing productive instead. I have been futzing around with some of the Haskell ecosystem. Mostly looking for something a little different to try instead of sticking in my usual comfort zone of Ruby. It’ll be nice to get back to doing some Haskell.

              1. 4

                I’ve been slowly learning After Effects for the last couple of months in order to do The Secret Lives of Data as videos rather than complex D3.js visualizations. My thought is to go with short 5 minute videos that are in a conversational style that explain basic use cases and overall structure of the topic. My first video is going to be on Apache Kafka. I’m crossing my fingers that this all works! :)

                1. 4

                  I’m looking forward to seeing this!

                2. 4

                  Hello, lobsters!

                  This week was rough, with some turbulence from the dayjob spilling over into my personal life. Fortunately it’s all resolved but it meant less work got done. Most of the hackery was bugfixes - paying for all the times I just shoved something out the door. The good news is that the current state of the product is where I actually want to keep things around! My wife loves the product and has started using it (and that’s how we found the aforementioned bugs).

                  Lessons learned:

                  • Address parsing is hard. So far I’ve got a bunch of manual rules that check for obvious errors (like the entire address entered as the street). Where would I look to find techniques to better parse addresses? Most libraries use an API and I’ve found those aren’t that great.
                  • Off-by-one errors abound, time zones are no fun.
                  1. 3

                    I am trying to add a “Send Invite” feature to Fire★ which creates an email ready to send your contact info to someone. I have it working for linux, but I need to get it working for windows and mac too.

                    I am also making the start screen more friendly to beginners so they know what to do to get connected. The problem with creating a p2p service with no central authority is user friendliness.

                    1. 3

                      Implementing step 1 of updating the urllib3/requests desired exception hierarchy - showing we can catch/distinguish between socket exception types: https://gist.github.com/kevinburke/b98e053a4bf9835c67bb

                      Reading this book on genetic algorithms in preparation for phase 2 of the custom roller coaster tycoon coaster project: http://www.gp-field-guide.org.uk/

                      (If anyone has book/resource recommendations for GA, or wants to collaborate on the project, let me know!)

                      1. 3

                        Nothing terribly interesting at work, mostly fighting to get something in production.

                        Outside of work, doing some research into object capability systems, trying to learn Dylan, and struggling to keep up with the AI planning course on Coursera that I’m weeks behind on.

                        1. 3

                          I’m working on a Kaggle competition. The goal is to classify pictures of plankton to determine ocean health.

                          I’ve gotten a decent start using an SVM classifier on cropped, resized images, plus their FFT as features. I’m a little discouraged by how long the algorithm took to run (around 8 hours, I think) on my MacBook.

                          I’d like to iterate faster, but simpler models aren’t doing very well. Adding more features isn’t feasible because of how long it’s taking to run. Not sure what my next step should be: dimensionality reduction, or learning to parallelize. I’ve never made the jump to doing “big data.” I work on problems that can be done on my MacBook, so I’m not sure where to go in that direction.

                          This is one of two problems I run into when I work on these problems. The other is that I extract all the features I can think of, augment the data in any way I can think of, validate several models and go with the best, and finally optimize hyper parameters. After doing that, I don’t know where to go, but I’m still only in the top 30% on the leader board at best.

                          1. 3

                            Last week I’ve spent most of my spare time preparing for a local programming contest. I made a slide deck about a few simple graph algorithms, and presented it to some of my co-contestants. I’ve also been toying with the Godot game engine. I’m currently working on a stress test to see how their built-in scripting handles large amounts of scripted nodes.

                            My extra cooling for the Xeon Phi has arrived, and it is loud. I’m going to try if my old configuration would be sufficient to cool it first, to try and keep the peace at home. I’m also hoping SLED or OpenSuse can work with the card, since I don’t really want to run and maintain yet another Linux flavour. (RH, CentOS and SLES are officially supported)

                            1. 2

                              Picking up on one of the projects I hack on sporadically, a wordlist generator with advanced query capabilities. (Currently well behind other, similar tools, but I’m having fun doing it.)

                              Goals for this week are to replace ledit with lambda-term (I’d rather not depend on things not in opam) and add some features to the scripting language.

                              1. 2

                                My impending move is looming, and we’ve started selling furniture we don’t want to keep and packing things into boxes. Three-year-old son is a bit bewildered about all this. Too knackered to do any coding outside work ATM (and it’s not like I’ve got time either) but have been able to watch a couple episodes of GoT. Also just started reading Neil Gaiman’s Stardust on my commute. Enjoying it so far.

                                Going to a Facebook’s London Mobile Forum on Wednesday. Think I’ve got a fairly good idea about what caused the accidental massive performance improvement I wrote about last week: It looks like a bug in the cache abstraction libraries that caused our caching to be less efficient than it should be. When I removed the abstraction, I removed the bug and we got a big boost of performance when under stress. We’ve deployed it to production, but because the production system is not under stress as a norm, it just manifests as lower load on backend & there is no corresponding improvement in latency. I suppose we may be able to save some money though.

                                1. 2


                                  Implemented jscs, flow and babel together in an isomorphic JS project. Still need to implement jshint with jsx/es6 awareness which should be a “drop in” sort of addition. So far, const is one of my favorite es6 features and flow’s static analysis is really interesting.

                                  Had a new hire start less than a week ago and doing more hiring/interviewing.


                                  Wrote another chapter for my snap book. I’m basing the new content around the unreleased v1.0 of the framework and thus run into some bugs occasionally. Docker makes it really easy to handle all of the unreleased and patched versions of various code bases though.

                                  Did a bunch of maintenence tasks on various production systems and code bases.

                                  I don’t seem short on ideas lately, just time to implement them. I’m trying to prioritize the snap book and scaffold/build out my other projects enough to put them on the backburner for now.

                                  1. 2

                                    $work: More on the news-article collection & analysis tools I’m working on. Lots of golang. We’re planning to use it to analysis coverage of the upcoming UK general election. We’ve got a landing page up now for anyone interested in receiving reports on said coverage, as they happen: http://electionunspun.net/

                                    !$work: finally started making some progress on releasing one of the games I’ve had kicking around for a while (ie 15 years or so). I even started a devlog over at Tigsource to keep me motivated. Actually, motivation isn’t the problem, it’s just finding the time to work on it.

                                    1. 1

                                      I’ve been so busy moving to Boulder that I haven’t had time to do much of anything else the last few weeks.

                                      It was exhausting, but the good news is I’m finally settled into the new place, and now have a much, much shorter commute, and a bunch more free time.

                                      Not computer related, but I’m reading “Criticizing Photographs”, which I picked up for $5 at a used book store.

                                      I also want to start a new project using Common Lisp, but I’m having a mental block coming up with anything.

                                      At work, I’ve been implementing some validation steps that use our logging database to provide feedback to our testing framework. I’ve also taken a few significant detours to make the infrastructure around our logging DB more stable.