1. 22

It’s Tuesday, not Monday, and I think we skipped a week but — what are you working on this week? Feel free to share links, ask for help, etc!


  2. 12

    As usual, I’m working with a language designed (originally) at Apple: Dylan.

    This week, I’m working on laying out a roadmap for improving our Unicode support. Unicode support was never finished in the Open Dylan implementation back in the 1990s when it was being commercially developed at Harlequin.

    This is turning out to be a more interesting subject than I’d thought, partly due to the release of Swift. In Swift, a Unicode character can be multiple of what they call Unicode scalars. You can iterate over strings in multiple ways (by character, by unicode scalar, by UTF8 byte, etc). I’m looking forward to finding time to install XCode 6 and play around with their string handling.

    The goal here is to create a list of small bite-sized tasks that give us an incremental path to Unicode support, so that it isn’t quite the intimidating task that it is now. We’re finding that there’s definitely a lot of work to do though just to get things rolling. (And more after that to clean up the language specification and work out how Unicode should really be integrated.)

    Other than that … I’m also still helping someone else out with some changes to our garbage collection interface and some other miscellaneous stuff.

    Edit: I’ve started putting the road map in the “Getting Started” section of https://github.com/dylan-lang/opendylan/wiki/Unicode.

    1. 11

      I’m finally back from 2.5 months in Asia, and it looks like a lot of core developers are interested in updating the Jenkins UI with a lot of components I used in my Doony chrome extension. So I will be working on getting up to speed with the Jenkins codebase and outlining a series of steps to try and go forward with a UI update.

      Super exciting!

      1. 3

        I wish you the best of luck as you descend into the pit of madness. I come from there presently, and it is a deep and dark place full of horrors, whence none should go if they value life.

        Mercy upon your soul, kb, mercy and luck.

        *thousand yard stare*

      2. 8

        Continuing work on my vector co-processor, Creek. I’ve gotten a very basic test of the datapath working. What I have left to do is more extensive datapath tests, the controller/instruction decoder, and FPGA integration.

        1. 3

          What has your experience with Chisel been? I’ve been doing classwork in Verilog and have been (as many are) underwhelmed by it. I’ve not used Chisel before, but if it is able to relieve some of the frustrations of Verilog development it may be something worthwhile to use.

          Of course, I would have to convince my professor to allow its use, but that’s another issue entirely.

          1. 4

            I’ve had a somewhat mixed experience with it. It is certainly an improvement over Verilog, but being a very new language it has all the problems that other immature software projects suffer from.

            The Pros
            • The language is designed specifically for describing hardware, so unlike in Verilog you cannot easily create a description which is impossible to synthesize. Also, the “standard library” provides easier ways to describes things such as shift registers, memories, etc.
            • There is a clear separation between registers and combinational signals. You will no longer be confused about blocking versus non-blocking updates. All updates to combinational signals are blocking. All updates to registers are non-blocking.
            • It makes it very easy to parameterize and script generation of hardware. This cuts down on a lot of repetition.
            • The verification environment is a lot better since you can write your testbenches in Scala instead of in Verilog. On Verilog projects, I would often write Python scripts or C programs to generate the output I expect from the module being tested and then copy-paste that output into a Verilog testbench. With Chisel, I can just write the code that generates the expected outputs in the testbench itself.
            • Each of your hardware modules is just a Scala class. If you have some specific modules you want to use in various different projects, you can publish them as a local sbt package. No more copying in random Verilog files from old projects. This is what I have done for the floating point units and crossbar switch I am using in Creek.
            The Cons
            • Error messages can sometimes be rather bizarre and unhelpful
            • Documentation is somewhat sparse. I’ve often needed to look at examples or in the Chisel source code itself to figure out how to do something.
            • There are still some bugs. I’ve discovered and helped fix a few as I’ve worked on this project. The Reverse function is broken in the latest release (2.2.x). If you need that feature, you will need to check out the master branch from github and install using sbt publish-local.
            • It isn’t as good at warning about width mismatches as I would like. It will happily accept an 8-bit bus connected to a 1-bit wire. Sometimes you want this, sometimes you don’t, but it would be helpful to be warned at least.
            • FPGA backend does not take advantage of board-specific functions like using Altera’s block RAM for memory. Not sure how I will work around this when it comes time to do integration.

            I’m sure the cons will gradually go away as the project becomes more mature. All in all it has been a much better experience than working in Verilog ever was.

            1. 1

              Have you used MyHDL, JDHL or another HL HDL language?

              1. 1

                No, I have not.

              2. 1

                Good to know. I may have to try it and out and let my professor know about it. Thanks!

          2. 8

            Two blog posts on recursion-schemes and teaching c_of_scheme how to inline toplevel functions properly.

            The second blog post is intended to be a massive wall of links/examples of cool uses of recursion-schemes, if you have anything like you’d like to see added, please let me know!

            1. 1

              Great stuff. Looking forward to reading.

              1. 1

                Ha, I came back to your blog just today to see if there was anything new on your implementation of c_of_scheme; I loved that article, and had a tab open with your latest post ready to read later today!

                1. 1

                  Thanks! I actually catching up on all my blogging so my high level over of c_of_scheme is done right now actually (I just typed C-xC-s). That should be up some time this week!

                  I’m mostly planning on writing about the compiler however, since well.. I know a lot more about the compiler than the runtime system :)

              2. 7

                I’m working on http://hackerqueue.io

                HackerQueue was a project for a hackathon I attended with a friend. We ended up placing third with the idea. I recently daemonized it, so the site should stay live. I’m looking for input about it. (I know there are some bugs, but if you break it please put an issue on GitHub. [Screenshots are welcome too, especially if it is hard to explain]).

                Also, please star it if you like it!

                1. 4

                  This is pretty sweet. You just compressed 3 of my browser tabs into one. Now for a feature request… maybe it is already there, but you need a way to go to the comments for that post for each site. Maybe a comments link next to the story to take you to the comments section for each respective site.

                  1. 2

                    Thank you for your feedback! I will definitely look into it.

                    1. 2

                      I love how clean the interface is, if you add comment links you may want to do it on hover for the desktop. Mobile needs another solution.

                2. 6

                  I finally have an answer for this! My day job doesn’t give me any opportunity to program, but I’ve managed to find some time this week to start developing a website for my State Emergency Service unit.

                  The problem we face is that we need to know the availability of our members to efficiently call up a team at any time of the day. Currently we’re using a service called Teamsnap which is designed for organising sports teams. I believe it’s not particularly well suited to our needs and has a lot of cruft we have no need for.

                  I’m developing my alternative in the Haskell web framework Yesod. Twitter’s Bootstrap project is making life even more exciting, though I’m somewhat dreading learning more about Javascript which will be necessary to get the interactive interface I’m trying to emulate from Teamsnap.

                  Fun times ahead!

                  1. 5

                    I’m working on leptus (my erlang REST framework) to implement a flexible logging system. The highest priority is providing a pluggable access log.

                    1. 5

                      Working on a high-performance metrics library for Rust. Emphasis on concurrent data structures and approximate algorithms. Been learning a lot about memory models and writing safe concurrent code in Rust…a lot of fun! So far I have an implementation of IntAdder and PCSA. Looking to add histograms and moving averages next.

                      1. 4

                        a HTTP API for Pandas called Koalas. More accurately, building a service layer around it to make available for public data hosting.

                        1. 4

                          “ask for help” reminded me of Adam Savage’s 10 Commandments of Making

                          1. 1

                            an excellent set of commandments :~)

                          2. 4

                            I finished the Clojure Koans and I’m almost done with the third chapter of Clojure Programming.

                            For $WORK, I’m finally past design and into implementation of my project.

                            1. 3

                              Working on my game idea code named TCFODS. The basic gameplay is now complete. The tween library works nicely. The graphics look horrible, but they are just placeholders (have to get gameplay working first).

                              This week I am adding particle generator (yeah, i will write my own because I am stupid). Maybe next week I will add sound. Then Have to start polishing.

                              I’ll post screen shots and gameplay videos once I get better looking graphics…

                              P.S. This is my first game using C++11 heavily. super super nice. I used C++11 for Fire★. But for game programming, it has become a language you can do rapid prototyping with.

                              1. 3

                                Just released an executable model of the Calvin distributed database (for my PWL talk tomorrow[1]):


                                Since Calvin’s implementation is not very accessible (only recently released and still incomplete[2]), this is the best way I’ve found to understand Calvin, and to use as a foundation for explanations. All concurrency and distribution is modeled in a single-threaded process with in-memory data tables, which makes it easier to see what is going on.

                                [1] http://www.meetup.com/papers-we-love-too/events/171291972/

                                [2] https://github.com/yaledb/calvin/issues/1

                                1. 3

                                  Going to be working on a postgres-as-a-service platform.

                                  1. 2

                                    that sounds super–when can we pay you for it?

                                    1. 1

                                      I’d be interested in hearing more about this, particularly in regards to tune-ability under differing query loads.

                                    2. 2

                                      I’ve been tinkering with learning Clojure via Tic-Tac-Toe, inspired by ZombieCLJ. Slow going due to lack of time, but fun :-) Since I’m going down the lisp route I’m going the whole hog and re-learning Emacs too.

                                      1. 2

                                        Went deeper on ElasticSearch and Cassandra last week, which yielded some great fruit. Learned a lot about Apache Storm 0.9 topologies and getting them to play nicely with Python, which continues my open source work on streamparse. Actually getting a fully-functioning analytics backend up-and-running with Storm + Kafka + Cassandra + ElasticSearch this week, I think, which is exciting!

                                        1. 2

                                          More stock trading stuff (mostly backtesting an automated trading strategy on historical data), plus a little bit of 3-D printing and modeling. In particular I just learned about 3-D modeling with interval arithmetic, which looks like an impressively useful technique for a variety of 3-D algorithms. Most of the work I’ve found in the area so far is in raytracing of implicit surfaces, but I was thinking it might be a cool way to render parametrically defined surfaces instead, so maybe I’ll do that.

                                          Where this comes from is that I wanted to 3-D print a continuous and fairly smooth heightfield standing on its edge on a RepRap, and so I took the simple approach of making two copies of the heightfield surface a millimeter apart, generating four surfaces around the edges to make it a solid, and generating an .stl file. Problem is, the thickness of the plastic depends on the slope of the heightfield. What I really want is something like the PostScript stroke or the <canvas> 2D strokePath operator, but in 3D — I would give it a zero-thickness surface and it would generate for me a 1-mm-thick solid enclosing it — and I don’t know where to find that. (One approach would be a Minkowski sum of the surface and a sphere, corresponding to “round” linejoins and linecaps, but that generates much more complicated geometry than is really necessary.)

                                          Really, it seems like there’s no equivalent of PostScript, or even SVG/PDF (PostScript without the programming language), for 3D yet. There are some cool algorithms in CGAL, including the Minkowski sum, and OpenSCAD is a little language with some parametric modeling features (including, again, Minkowski sum), but it still feels terribly limited and limiting to me. The OpenJSCAD variant, which uses JS instead of its own language, looks superficially more promising, and also has a way cooler UI, but I haven’t tried doing anything serious in it yet.

                                          So I’m getting together with a mechanical-engineer friend of mine tomorrow night who has done some pretty impressive 3D modeling stuff interactively, and also a little programming, and then hopefully I’ll understand a little better what’s out there now, and whether there’s a “3D PostScript” or not, and what it would need to do.

                                          Also, I need to get http://www.canonical.org/ back online.

                                          1. 1

                                            If you’re into Clojure, check out scad-clj. It’s somewhat actively developed and more powerful than plain OpenSCAD.

                                            Edit to add: more powerful in the sense that it’s very extensible, repl friendly, etc. You can still feel the OpenSCAD underneath, though. So it may still feel limiting. YMMV, I had fun.

                                            1. 2

                                              scad-clj looks like it’s generating the geometry in the same language that you use for writing models, so maybe I could successfully use it to do the kinds of things I want to do! Also, it has font support! Those are both super awesome.

                                              The main basis for my gripe with OpenSCAD is that your model program doesn’t have access to the geometry it generates, so it can’t do things like “smooth off sharp corners” or the shell operation I described above as an analogue to the PostScript stroke.

                                              Thank you very much for the pointers!

                                              1. 1

                                                To be honest, my 3D printing exploits have more or less been the digital equivalent of finger painting (unfortunately, I’m too old for it to be considered cute)–so, I’m probably not the best person to be recommending software for abstract geometric transformations that need to be correctly rendered in meatspace.

                                                I will say this, I really enjoy Clojure and it was a breeze to use this library to convert coordinate data* into printable form with this lib. You can set up the stock GUI (OpenSCAD’s that is) to listen for repl updates, so it’s painless to sculpt your model from your clj repl and compose functions and abstractions and what not–while watching your model update instantly.

                                                • I happened to have coordinate data as a subset of some data in a simulation I was writing. It seemed like a fun tangent to go and try to print some arbitrary abstract data. I think I had to write 3 or so functions to get a decently printable (and surprisingly complex) model up and going. From there it was hard to resist transforming the model from there.
                                          2. 1

                                            I’m working on a way for people that use gihub pages to no longer need to update that branch of their repo by hand, but have the full workflow automated.

                                            It’s presenting itself in a library called autopages and is coming along quite nicely.

                                            1. 1

                                              Working on the event counting part of my new behavioural anomaly detection product. Receive from Kafka, count things into buckets, generate a bunch of timeseries to pass along to the next bit.

                                              1. 1

                                                As usual, I’m working on my pet project: Myrddin. This week, I’m probably going to try to implement good pattern matching, with effective decision trees. I’m also going to be poking at getting more library code written, and maybe writing a self-hosting, dead simple build server in it.