1. 24

(Since no one else posted it yet…)

It’s Monday, which means it’s time for our weekly “What are you working on” thread! Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators?


  2. 21

    Today starts my contract to work on Rust’s docs!

    I will also be trying to finish up “Rails 4 in Action” ASAP.

    1. 4

      Are Rust docs and your work going to be in Sphinx?

      We’ve been using it for thousands of pages of documentation (including multiple books) in our Open Dylan docs and we really couldn’t be happier.

      We’ve got a custom extension for describing our APIs and for cross-linking to our language specification (which for legal reasons hasn’t yet been converted to Sphinx).

      I really think Chris Morgan got a lot right in this presentation but haven’t followed things much since then.

      1. 1

        I am not changing the tooling unless there’s a really good reason to. I am sympathetic to what Chris said, but I am mostly being employed to write prose, not code.

        We’ll see how it shapes up.

    2. 15

      As usual and like every other week, I’m working on Open Dylan.

      I’d written last week that I was working on learning more about our compiler’s typist and optimization systems. As part of that, I was working on getting some old internal tests of the typist working again. There was a problem where the compiler IR wasn’t getting set up correctly due to how those tests were running the compiler. I managed to fix that, so the tests actually run now. An interesting sidenote is that when these tests ran previously, the type inferencing wasn’t as powerful as it is now so many types are inferred more correctly and with tighter bounds than in the past. (In some cases, the type inferencing algorithm is now tracking the values stored in a binding rather than just the class.)

      I spent a lot of time debugging code last week and that’s always been moderately painful as our values aren’t directly visible or available in the debugger (LLDB or GDB). I decided that it was a good time to invest in writing Python scripts to integrate well with LLDB. As a result, I can now see Dylan values and classes while debugging:

      (lldb) frame variable
      (dylan_value) mepargs_ = 0xbfffd4c0 {<simple-object-vector>: size: 2} {
        [0] = 0x00000009 {<integer>: 2}
        [1] = 0x02861eb0 {<byte-char-file-stream>}
      (dylan_value) e_ = 0x0018a288 {<absent-engine-node>}
      (dylan_value) parent_ = 0x005cc384 {<incremental-generic-function>: do-execute-command}
      (dylan_value) T3 = 0x00000000 {}

      This required some changes to the C run-time and back-end, partly to make it more usable, partly to work around issues in LLDB. Some fixes have been submitted to Open Dylan and LLDB and more will be coming over the next week. There’s a lot of work to be done to implement better display for various types and to get changes to Open Dylan from being experimental hacks to something that can be safely merged.

      I didn’t expect to be working on the debugger and C run-time this week, but the effort is paying off in big ways already.

      I believe strongly in the value of making at least some progress every day. I’m now trying to communicate some of that daily progress better via the @DylanFoundry twitter account.

      1. 7

        I ran into a major roadblock with Creek, so I’m taking a step back from it and moving on to different projects.

        As some of you may have seen, I’ve been trying out implementing a Lobsters bot to automate repeating tasks like posting this thread every week. I assure everyone that I have no nefarious intent.

        I’m also starting to play around with Rust. I’ve already built the compiler so that I can cross-compile for the ARM chip on my FPGA board. I think I might try to write some bare-metal code and get it running on the board.

        Other than that, I’m working on a set of lessons in Javascript programming for Seattle CoderDojo and thinking about starting a Chisel cookbook to document some common patterns and best practices for RTL design in Chisel.

        1. 6

          NixOS and Vala as usual for my free software projects. At work, I’m working on my second FUSE file system. We will probably open source the first one. It’s a star-topology (also other topologies available) replicated master-master active-active file system.

          1. 4

            What are you doing with NixOS? I’m a big fan and use it everywhere I can.

            1. 1

              I have a NixOS server, mostly for personal stuff or rebuilding packages instead of doing it on my desktop, and a couple of public services. About contributing, I’m maintaining gnome 3 and some other stuff.

              Not yet using it as my primary desktop just because I’m still planning how to install nixos in a subdirectory /nixos of my existing debian desktop, and chainload grub. That way I don’t have to backup the data and redo everything in order to replace my debian installation.

          2. 6

            This week I’ll begin porting my current MVP to our new stack based on Erlang (Elixir) and the JVM (Scala)!

            The first thing we will be building is a new Erlang RethinkDB Driver based on their new JSON API, of which we will base much of our platform on. We will also be building services for ID generation, service registry, and more, all of which will be open sourced in coming months.

            1. 1

              Curious why you are using both erlang and jvm in a new stack. What tradeoffs went into the decision to use a mixed architecture like that?

              1. 4

                The original prototype was built in Node.js, which worked well for a prototype, but not much else.

                We chose Erlang and Scala for several reasons:

                The kind of application we are developing (a realtime social application) preforms much better on Erlang’s process model, especially when it comes to concurrency and error handling, among many other benefits. The original plan was to build the entire stack on Erlang, but after some investigation with our test data, it was quickly shown that the JVM well out-preformed Erlang when it came to jobs like map reduce and machine learning. If we were building the service as a monolithic app, we would have stuck with Erlang for everything, but as the service is being built as a service-based architecture, we decided to build our processing nodes in Scala (chosen over Java mainly due to syntax and actor support).

            2. 5

              I made progress on everything!

              I’ve finished chapter four in Clojure Programming. I also have started to develop my usual “Hello world” app with compojure.

              I’ve also made progress at $WORK. They decided my design work was out of scope (sadface), but in terms of practical progress, I’ve made it through a couple levels of abstraction.

              1. 4

                Last week I was working on trying to embed a gtk+ widget in a tree of NSViews, and this week I’m still working on this. I made some progress on enter/leave events and mouse button events. And right now I believe expose events are correctly recursing down into these gtk+ widgets that are otherwise disconnected from others, but as far as I can the backing NSView isn’t correctly being told when it needs to be updated. So while expose events seem to work when they are generated, they’re not yet generated in all the places they should be.

                1. 3

                  I’m working on a post about loading data into Cassandra. CQL has made some types of bulk loading more complicated. I’m hoping someone else will find it useful.

                  1. 3

                    This week, I’ve been involved in the discussion for a CLOS-analogue for LFE as we discuss various approaches for building this (taking a lot of cues from PAIP). I’ve yet to provide any pull requests yet, though.

                    1. 3

                      While most of the time I still spend it on OpenHorus, I’ve been doing side-stuff that may or not be related.

                      For example: Unity3D’s NAT facilitator went down (it still is? I don’t know) so I had to put a copy of that on my vps.. sadly it didn’t compile on FreeBSD.. because Raknet 3.x is a complete mess!

                      Here’s some of the stuff you may find on the Raknet source:
                      Drunk code - Legacy code inside #ifdef mazes - More random #ifdef - Missing “returns”
                      Double includes! - Completely commented out files.

                      There’s more stuff like code that behaves in a very different way depending on the platform (RakSleep could be an example).

                      From what I’m seeing Raknet 4 has been cleaned up, but Unity and whoknows how many people are still using 3.x.. argh!

                      It compiles now, if you’re doing a Unity project and want to host your own facilitator, check out my fork, it still doesn’t compile on clang, but at least your nat facilitator won’t have a SMTP client linked to it.

                      1. 3

                        This week, for work tasks I can talk about:

                        I’m interviewing a couple people and trying to get a few more in the door.

                        Best way to love the team you are on is to recruit them. I’m also working on rewriting a storm topology that is need of “modernization”.

                        And we have a couple days we’ve set aside for working on open sourcing some of our stuff and documentation that is always a nice change of pace.

                        And outside of work:

                        We’re working chapter 7 of Storm Applied which has turned into devops 201 as it revolves around Storm topologies & resource contention.

                        1. 3

                          I am continuing my work on Kaiju[1]. I have a local branch that is not pushed which contains much of the infrastructure needed to listen on a TCP socket and push work to an underlying backend. From there I’ll have a platform for playing with different backends and building new features on top. I’m hoping to have a complete infrastructure with minimal functionality in the individual components done by the end of the week. And then build out the minimal components.

                          [1] https://github.com/orbitz/kaiju

                          1. 3

                            Playing with Scala and Akka, finding and creating spaces for nomadic hackers to go, enjoying the south Italian sun, hacking with a group of others on distributed personal data.

                            1. 3

                              This week is more wrapping up of the automation I’ve done for our entire infrastructure to be driven by ansible.

                              It’s been nice not only automating the configuration of everything, but automating the interdependent parts. For example, collectd/carbon/graphite are automatically populated with many metrics and dashboards tailored to the # of servers and their roles in each (ansible-defined) cluster. That means each dev environment also has a functioning collectd/carbon/graphite installation just like production does =)

                              Having most tooling we use be nice and unix-ey and file and sqlite-driven has helped immensely with automating it all.

                              Also, I’m continuing to smoke-out issues in our new FreeBSD cluster on EC2, which have been minimal, hooray!

                              1. 3

                                More hacking on the backup program I started on (Pirate Onion. The program that arrr-chives your data.(https://github.com/oridb/pirateonion))

                                And the accompanying yak shaving involved in fixing language bugs.

                                1. 3

                                  Setting up for a very early alpha demo of my Behavioural Anomaly Detection product tonight, then investigating porting Oleg’s Typed Tagless Final Interpreters technique to Scala for an expression language DSL.

                                  1. 2

                                    Actually working on Fire★ some more. I did more bug fixes and added checks to make sure the people in your conversation are in a clique. Will add some UI to inform the user if a person cannot be added and give an option to introduce them to others in the conversation.

                                    Will also work on my secret game called TCFODS. I added sound! Now to add a start screen, score screen, and level screens.

                                    1. 2

                                      I’ll be at MongoDB World with the rest of the TokuMX team. If you’re in town, come say hi! We just released TokuMX 1.5 last week (nice stuff for time-series applications in there), and a brand new docs page that I’ve been working on for about a month: http://docs.tokutek.com/tokumx.

                                      1. 2

                                        I’m trying to implement C#’s async/await with sweet.js, a macro compiler for Javascript. Also, I’m writing emails to ask for interviews for Debuggers, my blog about interesting bugs.

                                        ps: I’m sure you guys have lots of cool debugging stories to tell. Send me a message if you’re interested in an interview.

                                        1. 2

                                          Writing a course and set of posts on debugging production web apps. Covering stuff like common shortcuts to fixes, methodologies for dealing with the really hard bugs even though your hair is on fire (e.g. differential diagnosis), using metrics effectively for debugging and so on.

                                          I’d seriously love some proofreading/feedback from folk who run (or have run) server software in production.

                                          I’d also love to hear/read your favorite production debugging stories/postmortems - I want to compile a big ass list of good examples of that kinda stuff somewhere.

                                          1. 2

                                            I’m going to fix up some issues with the lobste.rs android app link.

                                            Notably fixing up links to ask questions & hopefully add collapsible comments.

                                            1. 1

                                              for work I am working on getting version 0.1 of my internal Dev Ops tool ready so people who need it ASAP can start using it.

                                              for fun I am working on developing a website that interacts with both the Twitch.TV API and Steam API to help me decide if I want a game. And I’m also considering rewriting my site, http://hackerqueue.io, in Angular.JS since it would probably be more efficient than the jQuery it uses right now.