1. 24

It’s Monday, so, what are you working on this week? Feel free to share links, ask for help, etc!

  1.  

  2. 15

    As usual, I’m working on Dylan, a language once designed in part by Apple.

    Last week, we put together a good start on a roadmap towards Unicode. We’re still looking for help with that work, and a lot of things can be done with small amounts of time on a regular basis (an hour or two or four per week).

    This week, as part of our drive towards evolving the language further (Dylan 2016), I’m digging into compiler internals much more extensively than I’ve done in the past. In particular, we’re interested in how we can bring the work that was done for a paper, Extending Dylan’s type system for better type inference and error detection, which brought function types and parametric polymorphism to Dylan, into our master branch. The previous implementation is useful as a guide, but was too slow for production usage.

    I’m also assisting someone else with compiler modifications to produce warnings for unused variables. This has involved learning a good bit about the internal data structures used for the control and data flow graphs. It has been pretty enjoyable and definitely shows where we need to improve the documentation of our internals. We’ve also found that source location tracking is incorrect for let bindings (let x = 23), so we’ll want to fix that before enabling the warnings about unused variables.

    Finally, I’m going to be submitting a patch to Sphinx this week that improves the display of the ePub output in iBooks. iBooks doesn’t display font-related CSS by default on many elements, which makes technical works a bit more difficult. We just need to emit a bit of metadata to tell iBooks to stop ignoring the font-related CSS and all will be well. I’m also working with someone else on submitting a patch to doc2dash to provide support for Dylan documentation generated by Sphinx.

    As usual, we’re always looking for help and we have plenty of things, big and small, where we could use assistance. :)

    1. 10

      A parser combinator library! for Swift, called Jiffy. I got a Doge Serialized Object Notation parser to typecheck, but now I have to implement the combinators for real instead of relying on stubby stubs. I wish the beta compiler didn’t spit out “unimplemented feature” errors so much; this would be a lot shorter with recursive enum’s but, all well, six month’s time. https://github.com/hlian/jiffy

      1. 8

        Over the weekend, I added a bevy of new features to Glisp, my LISP dialect written in Go (it has macros and syntax quoting now … yay!). This week I am continuing work on my vector processor Creek. I’ve put together the top-level module, but now I need to validate it and then figure out how to run it on the FPGA.

        1. 7

          A number of tasks, mostly involving Storm and working on Chapter 7 of Storm Applied but, the most important thing I’m doing this week is finding a new member for our team here at TheLadders. So, I might as well talk about that here as well. We wrote it up for “who is hiring” on HN:

          https://news.ycombinator.com/item?id=7829066

          You can also learn more about the team and myself in this lobste.rs interview:

          https://lobste.rs/s/gheyhy/lobsters_interview_with_sean_t_allen

          1. 6

            Trying to understand the algorithm in this paper for computing discrete logarithms in small characteristic.

            1. 6

              I’m probably not working on this again until the weekend, but I’m kind of excited about it so here goes:

              I’m working on fixing Mono’s FileSystemWatcher class on Mac (and BSD). We noticed this when trying to play with Microsoft’s ASP.net vNext on Mac. The kqueue implementation didn’t handle subdirectories, so I’ve been fixing that. I’ve got it basically working I think, but I need to compare it the native Windows implementation to look for edge cases, then write a test suite for FileSystemWatcher.

              This is my first time to contribute to Mono itself, so I’m kind of excited by it. :)

              1. 6

                I am learning how computers talk to each other. Why? Because I don’t really know how it works. I’ve been interested in networking in the past, but I’ve mostly focused my personal learning on programming languages. That sounds a lot more interesting than it really is, as I don’t think I know enough about the topic at this point to say that that was my “field of study”. Nonetheless, it was my interest and I want to move on. I’m starting with the basics of networking and building up from there. I’ve been doing a lot of reading, and am looking for a good place to dive in. Any suggestions would be appreciated.

                1. 2

                  I found it really beneficial to use packet and flow capture utilities to understand the structure and sequence of the network stack. Look at IPFW on Mac/FreeBSD, you can even redirect certain flows through user space and back again which allows you to write little scripts that analyze or transmogrify the packet stream. Wireshark supports writing scripts in Lua.

                  1. 1

                    Thanks!

                2. 6

                  Working on data structure / use-case driven cheat sheet for Haskell.

                  Got a positive response to this video: https://www.youtube.com/watch?v=Li6oaO8x2VY so I’m thinking about possibly doing more of them. Particularly now that I have a less terrible microphone.

                  1. 6

                    This week my goal is to finish the minor changes I’ve made to Dylan for it to cleanly compile and run on ARM Linux. Once that is done, I’ll see if any further changes are required for it to play nice with the Android NDK and get a Dylan “hello world” running on Android.

                    1. 5

                      For fun, I’m 180 pages into Clojure Programming, halfway through the chapter on concurrency and parallelism. I’ve also started looking into porting my version of a “Hello world!” web-app to Clojure.

                      For $WORK, I’m frantically trying to get a handle on what I’m doing and make real progress. There’s a lot to learn when working with an enterprise codebase.

                      1. 5

                        I’ll be continuing work on porting the PureScript compiler to PureScript, after a small break. Specifically, I’ll be working on porting the interactive mode, psci.

                        1. 4

                          I guess I’ll start it off. I’m working on learning Angular.JS for building an efficient way to interact with a mongoDB for work. (Most my coworkers are Java freaks so I volunteered to do the web dev).

                          Edit: I’m also looking for a JS library that is in the development phase to help contribute too, and trying to find time to change http://hackerqueue.io to Angular.JS instead of mainly jQuery.

                          1. 1

                            Just a small cautionary note, we had exactly one system written in Angular at a previous company, and there was a fairly steep learning curve, which hindered contributions from the rest of the company.

                          2. 4

                            I am working on Datazenit, web based tool for database administration and management. It features visualizations, rich content editor, schema builder and so on. It is being built on top of Scala, Play 2, Backbone.js and D3.js.

                            Screenshot #1
                            Screenshot #2

                            1. 3

                              I’m trying to learn Ansible to automate my personal boxes. Any tip? Besides that, I’ll be writing emails all day long…

                              1. 3

                                I made https://github.com/al3x/sovereign for my personal VPS, then open sourced it. It’s since grown into something pretty full-featured. Even if you decide not to use it, we use many features of Ansible so the source should be informative.

                                1. 1

                                  That’s really cool, I can’t believe I haven’t found it on my research. Thank you!

                              2. 3

                                For work, three main projects currently: Writing more parts of the Aggregations chapter in our Elasticsearch book, polishing up a PR for query profiling, and working on a prototype to identify similar time series using wavelet decompositions.

                                Rust is sucking up all my free time. Working on a Disruptor-inspired message passing data structure. Started as a director port of Disruptor, but has slowly migrated to a new syntax and different abstractions. Still the same idea in spirit however.

                                1. 3

                                  Work, Work, Work. And doing some work on refactoring a game idea Ive been toying with for a while.

                                  1. 2

                                    Enjoying a moment of ‘play’ time with my current client, working on using Docker to do a map-reduce-alike on our Django test suite, so that we can run lots of tests in parallel. (At the moment, the entire suite takes 20mins to run…)

                                    1. 2

                                      Trying to get some regular sleep as the child is booting up.

                                      • Getting micropython running on my STM32F4
                                      • Playing with Chisel and learning Scala at the same time
                                      • Reading a bunch of papers on spherical and hyperplane hashing
                                      • Playing with some discrete event simulation code in Lua
                                      • Sewing up some fleece swaddles
                                      1. 2

                                        Playing with Chisel and learning Scala at the same time

                                        Feel free to reach out to me if you run into any issues with Chisel (the errors it gives can sometimes be rather cryptic). Just as a hint, never call your module parameters width. That one made me waste several hours of debugging.

                                        1. 1

                                          Working my way through the demos and using it in anger before asking for help. Are there any structural visualizers for the AST or netlist as understood by Chisel? Do you use IntelliJ?

                                          I still remember the time I made a program called test and couldn’t figure why it just kept returning w/o producing output. Finally figured out after I deleted it!

                                          1. 2

                                            Using it in anger is a pretty good description of some parts of Chisel development, unfortunately. This is largely due to the lack of good documentation. I don’t use Intellij, but if I can give you some advice on how to make the development process less painful by playing around with the flags given to sbt.

                                            • To answer your specific question, you can generate a Graphviz file showing the connection graph of your hardware by using the Dot backend (pass the flags “--backend dot” option to sbt run). However, I haven’t found this to be particularly useful for making sense of the generated hardware. And for larger designs the dot tool will choke trying to produce pdf or svg output.
                                            • If you want to get a sense of what hardware is being generated and you can read Verilog, I suggest using the verilog or FPGA backends (“--backend v” or “--backend fpga”) and inspecting the generated verilog file.
                                            • If you want to see a timing diagram for your test bench like the ones Modelsim produces, pass the “--vcd” flag to sbt and you will get a .vcd file that you can open in GtkWave. I’ve found this to be the most useful debugging technique by far. However, know that the way it displays register updates can be … misleading.
                                            • Finally, the compiler by default does not export the names of your internal combinational signals. However, you can make it do this by passing in the “--debug” flag.

                                            I really should start writing a Chisel cookbook to document all the techniques I’ve learned, but I want to finish up Creek before I start doing that.

                                      2. 1

                                        PHP, Image galleries, PDF invoice generators - booring stuff.

                                        1. 1

                                          At work, preparing to open source an SSL project. Outside of work, I’ve been reading “Networks, Markets, and Crowds” (it was a book for a Coursera course a while back, but I never got around to reading the book or finishing the class) and exploring the ideas in the book with Racket.

                                          1. 1

                                            $work: Retooling a bunch of things, focusing more on one of my hats, to the exclusion of the others.

                                            !$work: Continuing to work on katuv, my DSL-DSL for ruby. I’m reworking some of the ‘code generation’ bit, it’s on the home stretch now, but it’s hurdles all the way down. In the meantime I’m also probably going to be doing some transplanting in my little window-garden. My Kidney Bean (I’ve named him “Jim”) needs a bigger pot, and the Cilantro needs to get split (I overplanted one of them). Fortunately, the Zinnias are doing just fine in their makeshift pot for now. Protip – if you want to have a nice green thing on your desk (well, assuming your HR vultures allow it / you work remote, like I do) that’s low maintenence, most beans (kidney, lima, etc) are super easy to maintain and grow just find in lower light conditions. Just keep them watered and turn them about 90-180 degrees every day (so they grow mostly straight). Only minor pruning needs to be done to keep leaves from getting shaded out. A bit of light gardening is a great way to reduce stress, too, which means you’ll be more productive.

                                            1. 1

                                              I’m working on a new blog post about using Lua scripting in Redis to implement analytics algorithms. Also working on an internal Go app to better manage software defined networks.

                                              1. 1

                                                Improving the visualization mechanism and documentation for Automat, which is an FSM library based on Ragel, but constructs the automata via combinators rather than a DSL.

                                                1. 1

                                                  A renderer-agnostic game engine in Go! Trying to work concurrency into a good shape for threaded game logic while keeping rendering on another single (possibly parallel) thread (most renderers don’t like multithreading)

                                                  1. 1

                                                    Adding multidimensional time series to my event counting service. i.e. when events arrive with { dataCenter: Foo, systemType: Bar} I want to generate time series for every inhabited cell in the cartesian product of (dataCenter × systemType).

                                                    1. 1

                                                      I would recommend that you not use this in production unless you have a way of restricting the set of valid things that people can insert. Every key must have a small, finite number of possible values, and product of the cardinality of the keys should also be small. Consider the case where there are twenty different possible values for four different keys. Suddenly, you’ve created 160,000 timeseries. Slicing and dicing them can also quickly become difficult, especially if people want to be able to select over an arbitrary set of them. One way to help yourself out here is to pre-aggregate on every key. Add a special * value that is the sum over every value for that key. It’s expensive space-wise, but worth it if you have enough data the querying could become a problem.

                                                      Another thing is that users who don’t understand your system will create keys that have values that are business ids, like “user id”. This can quickly explode your tsdb.

                                                      I think it is a reasonable decision to just not support tags in a TSDB, but they can definitely be nice for some things.

                                                      1. 1

                                                        Hey, thanks for the thoughtful input. I’ve been thinking about these issues too, and I’m using an LRU cache to cap the number of discrete “cells” we’re counting in any particular time series class to the top-K most recently seen combinations. At the moment I’m not writing anything out, it’s purely streaming, in-memory. But sometime this year I’ll be adding long-term storage, so we can do fun stuff like history-aware trending analysis and back-testing of speculative algorithms. :)