1. 25

It’s Monday, which means it’s time for our (semi-) weekly “What are you working on” thread! Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators?

  1.  

  2. 12

    I always love reading these threads, but with work and life I rarely have anything to contribute. However work has settled down recently, and I’ve finally had time to get back to a long-time project: a math expression visualizer. (demo of its current state). If I have time, I’ll be adding: a play button to the sliders, so you can have it animate against any independent variable; graphs across independent variables (lots to experiment with!); dynamic scaling of the bars chart.

    I would love to hear any feedback, ideas or suggestions!

    The first thing I’ll be tackling is actually the app architecture. I still have some race conditions and some ugly hacks, as I’m just figuring out how to use this cool library called reflux. Possible blog post coming about the challenges I had, coming to it from Backbone.

    1. 4

      my comment is i have absolutely no idea what it’s visualising …

      1. 3

        Would you mind explaing a bit more on what the Maths expression visualizer does and how it can be used? Thanks! It looks incredibly interesting, thus my question.

        1. 3

          Useful feedback, thanks! I have only shown it to other people in person before this post, so I’ve been able to give the explanation right there. I don’t know how to do that in the app itself yet… it’s hard to know the balance between a long essay explaining everything, and just giving enough hints so that it can be explored and discovered instead. I’ll give my first shot here (edit: moved to a gist to avoid clogging the thread with an essay…)

          1. 3

            Ok, reading through it now, will post comments and feedback afterwards.

    2. 10

      I took off until the new year to watch our baby. In between that, I plan on livecoding a bunch. Some stuff I plan to broadcast:

      • Learning OCaml or Rust
      • Exploring microservices, probably with RabbitMQ and Thrift
      • Working on some CLI utilities in Go

      I’ll post when I’m broadcasting to http://twitter.com/justincampbell.

      I’m also pairing with people on Fridays. If you want to pair, contact me on Twitter!

      1. 9

        I do Haskell stuff.

        1. Twitter NLP bot

        2. Github events -> IRC bot

        3. Book, g'dammit.

        4. Talking to publishers about #3

        5. PR’d the twitter-types Haskell library, probably continue to kick that around as #1 develops.

        6. Hoping to avoid Haskell infra work this week (Hackage, Cabal), but it seems like I might end up fixing Hackage-Fu.

        1. 4

          github link for the twitter nlp bot?

          1. 4

            It’s still very, very early. It’s a project for a student to learn NLP + Haskell + API access + data persistence.

            There’s nothing interesting yet. Rest assured, it will be posted to lobste.rs when it has expanded a bit. We’re still figuring out how to pull down and analyze data.

            Also, it’s a bot obsessed with pie. Just pie.

            To give you an idea of how early: we haven’t even tested a dumb, manual scoring lexicon with word normalization - let alone a topic model / classifier.

            1. 3

              There’s nothing wrong with being obsessed with pie.

              1. 2

                What kind of models are you going to try?

                1. 2

                  Start with dumb normalization + lexicon with manual scoring, then move to trained classifiers probably. If you have alternate suggestions I’m open to them, I’ve got a bucket of links about topic modelling I’ve yet to go through fully.

                  1. 2

                    Nothing in particular. I’ve done LDA topic models many times, but have never thought about how to operationalization them in a Twitter bot.

                    1. 2

                      what?

          2. 8

            I’m trying to write a Twitter bot obsessed with pie (in Haskell). It is…slow going, but I am learning a lot. This week I am trying to figure out acid-state and how to test to make sure what I have so far is working as it should. And someday if I keep poking at this thing, I will bend Twitter to my pie-loving ways.

            1. 6

              Over the weekend I finally charged my drill and started putting shelves up around my television for my media stuff. (Up until now it’s been the TV on the wall, and a TV stand on the floor with consoles, server, etc on it. Putting it all on the wall to free up some floor space and tidy up the cabling!) This lead to me prototyping the shelving placement by screwing directly into the plasterboard temporarily - https://twitter.com/Caius/status/539475214204411905 has a picture. Which I found way more amusing that I should have done.

              Need to continue on, buy some brackets and cut some wood for the shelves this week. Also finally gotten my head around ansible & puppet it seems, so I’m continuing to get a couple of vagrant servers set up with recipes to finally have reproducible builds for my personal web servers & dev machines. (Very interested in replacing my shonkey homebrew-install script with something better for starters.)

              Also looking forward to the end of my week, even if it does involve leaving at 0500 Sunday to drive halfway up the UK, but it means I get to go cycling with some colleagues around Edinburgh where I’ve not cycled before.

              1. 6

                At work, working on some internal SSL infrastructure.

                At home, I had a weird desire to get back to embedded development, so I’ve started a harebrained project to write my own on some ARM boards I have. My background is mostly in MIPS, so this should be interesting, for various definitions of interesting. This week will be getting up to speed on ARM assembly and working through some basic tutorials to remember the dark arts.

                1. 3

                  I did some ARM assembly in an Computer Architecture course last year, we ended up implementing a mini jvm, which was really fun!

                  1. 3

                    I worked as an embedded engineer for a while, so I got to do some MIPS assembly there and I agree — it’s a lot of fun, in it’s own strange way. Leaving embedded development has led to some other interesting opportunities, but I still miss it.

                2. 5

                  I’m taking a week off, so I’m going to work through Functional Programming in Scala, and see if I can get my feet wet with some aurora stuff.

                  In particular, the problem that I’m interested in is speeding up our rolling restarts. We have pretty strict SLAs, so we have trouble restarting big batches of machines simultaneously (for our really big servers we can do a few percent at a time). I’ve been working on making our clients more resilient to rolling restarts (you can see some of my work in finagle) but I think the really big gains will be in changing the way our scheduler runs the rolling restart.

                  Caveat lector: I am not terribly familiar with Aurora, so this may be full of lies. If you know better than me, please correct me!

                  There are two main problems. The first is that we want to minimize the number of simultaneous unhealthy machines, and restart at such a rate so that we only have enough unhealthy machines to impact our success rate so that it stays above the SLA. The second is that we don’t have a good metric for healthiness. Aurora is able to query a health check, but all that tells us is that the server has stood up, which is nice to know but not as useful as something like, “I’m alive, my port is open, I have connections to my downstreams servers, all of my lazy code has been evaluated and I’m all JIT’d up.”

                  The proper solution to both of these problems seems nontrivial.

                  To solve the first one, you would supply an SLA, and a tolerance for how close you can get during a deploy, and then make sure that you never dip below that rate, by being able to directly examine an SLA for a service. This entails hooking up to our observability system, which would be feasible, because it has an API, but also introduces an extra moving part to an already complex system, and would be a significant undertaking.

                  The second one is hard to do correctly regardless–we could do a cheaper one which is, “has your port stood up” which might be feasible, but the current health check is probably a reasonably good proxy anyway. It might take a few seconds after that health check to actually get in a good state, but that’s OK.

                  Aside from perfect solutions, I have a couple of things I want to tackle which should improve the situation. Aurora has a configuration where you can say how long you want to wait before doing a health check. Aurora will then wait for that length of time before doing your health check, and if any node fails, it restarts the batch. This means that people choose very conservative timeouts, in case there’s even one machine that’s slower. If we could short-circuit by checking every second, then we could move on much faster. That seems tractable for an Aurora noob.

                  The other thing I’d like to tackle seems a little trickier. The problem is really that we don’t want to have more than X machines restarting simultaneously, which technically batching achieves, but could be done much faster if we restarted a new machine as soon as (or with a few seconds delay to account for our crappy health check) one of our machines had restarted.

                  Imagine that we want to deploy four machines, and we can have at most two unhealthy machines at a time. Here is what it looks like today:

                  ==>
                  ========>
                  ……………………………………………………..==>
                  ……………………………………………………..==>

                  Here is what I would like for it to look like

                  ==>
                  ========>
                  ……==>
                  ………….==>

                  This may turn out to be way too ambitious. I haven’t touched Aurora before, so I don’t know the codebase at all. Let me know if you have any advice.

                  1. 5

                    Last week, I hit a whole month on my github streak. I decided that I was already more than halfway to my own JavaScript transducers implementation, so why not muddy the waters a little with my own implementation: transduce and used it to back underscore-transducer. (NIH, I know, but it’s in fun.) I also made the “Most Prolific” section on npm during that transition, since transduce is based on small standalone libraries. Something I never thought would happen, to be honest.

                    This week, I need to start preparing some material for a JS user’s group meeting (on transducers, of course, since that’s all I ever do for side projects any more). Also, I’ve discovered that lenses are probably very interesting to me, if I can ever wrap my head around them.

                    1. 5

                      As you may have seen on here over the past few days or so, I’ve been revising for some phone interviews with Google. It’s been a fantastic excuse to learn about all kinds of really cool stuff. Thing I’ve learnt/practised about include:

                      • The Trie datastructure (radix tree), here’s a nice explanation
                      • Detecting cycles in LinkedLists using Floyd’s circle finding algorithm
                      • Implementing most standard datastructures and sorts (Heaps, HashMap, LinkedList, ArrayList, BinaryTree, QuickSort, MergeSort)
                      • LRU cache stuff + implementation
                      • Ways to implement a Bloom Filter and why it’s useful

                      There’s more stuff too, but that’s the stuff off of the top of my head that I enjoyed learning about.

                      1. 5

                        I work on a retail SaaS product, so I’m watching my scale planning and testing bear fruit. Currently pushing over a million requests per minute to over a million concurrent users. I haven’t thought about the rest of my week yet.

                        1. 5

                          So far this week I’ve submitted my first pull request to GHC:

                          https://phabricator.haskell.org/D547

                          The motivation is to make read . show ≡ id for Data.Fixed. The QuickCheck property now passes! Sadly the fix is to change the Show instance, I have no idea how large of an impact that is for people.

                          1. 5

                            I hope to finish ripping out the old PureScript lexer and replacing it with the shiny new lexer from purescript-in-purescript, which has better support for optional indentation and comments.

                            If there’s time, I would like to start working on a small showcase project I have in mind for Haskell and PureScript.

                            1. 3

                              I’ve also started working on a daily blog series: 24 Days of PureScript, in the style of Oliver Charles' wonderful 24 Days of Hackage.

                            2. 5

                              Reading Purely Functional Data Structures by Chris Okasaki – only around 200 or so pages – can do in a week. Also going through Learn you an Agda.

                              1. 4

                                I’ve been slowly working through this in Haskell, as something to work on when I need a break from other things. This also gives me plenty of time to think it over.

                              2. 4

                                I’m working on a tiny versionning webservice. It stores diffs for posts to a given url paths, and allows for easy retrievals of previous versions of the document. It’s not exactly super pretty nor super efficient, but I’m still happy about it. It seems clean enough for use, and is well under a hundred lines of readable code. So yeah. Happy.

                                Next step is integration with some type of in-browser editing software, maybe. As that’s all in my free time, and since there’s very little of that, I’m taking baby steps :P

                                1. 4

                                  Last Week

                                  I completed the API reference for Fire★ which you can see here!

                                  I also updated the gui to use fancy icons instead of text for most buttons. I think it looks nice. I wrote a blog post about it.

                                  Wow, thanksgiving break was productive!

                                  This Week

                                  Today I expanded the API and made some bug fixes to the app editor. My plan is to start expanding the API more this week, particularly improving the draw object

                                  If anyone wants to help expand the API, I would appreciate it. But most of all I need testers!

                                  P.S. I like to put what i did last week as well as this week because I typically post every week. My end goal is to see how well my plans match with what i do.

                                  1. 4

                                    Rereading Steve Blank’s works and reading Dr. Pfenning’s lecture notes on Linear Logic.

                                    1. 3

                                      Last week I got very little accomplished, partly because of Thanksgiving holiday. This week at work I’m doing some more bug fixing around gtk+ and monodevelop.

                                      But on the side I’m trying to learn more about Flux/React.js and how to integrate it with a Rails app.

                                      1. 3

                                        I am grinding through the doc/config tasks to package a gem so that Rails sites can easily support FIDO’s open two-factor authentication standard.

                                        And it really is grinding. The code has worked for weeks, but all the fiddly bits of packaging a gem, making a Rails engine, making it configurable, writing an intro that makes sense if you haven’t been staring at the FIDO spec docs for two months… it’s been a real reminder of how much work goes into a project besides the code.

                                        1. 2

                                          Isn’t that the truth? I’m not sure why, but I’m consistently surprised by how much work ends up being polish, documentation, and developer experience.

                                        2. 3

                                          Slow going, but making steady progress on my tic tac toe implementation to improve my TDD and golang skills at the same time. I think I’ll have it done this week, and then on to a new mini-project. Possible candidates:

                                          1. do something with my parallela.
                                          2. start some code katas
                                          3. another small programming problem

                                          The intro to the command line series took a week off, but I’ll be writing a few more entries for that this week as well.

                                          1. 3

                                            I spent the long weekend visiting family and skiing, so didn’t get much done that’d be of interest here.

                                            At work I’m adding some logging to help us detect a few tricky EIO error cases. We’re going to have a utility that will show us how often the systems in our test lab have EIO conditions (for example if failover takes longer than it should), and will narrow down exactly which failure cases are causing the EIO. Hopefully, it will also help the on call developer and support staff debug customer EIO errors.

                                            Outside of work I haven’t been doing much code related. This week I’m going to make it a point to update my new blog before it dies off like the old one, so I’ll probably starting a new project or cleaning up and uploading an old one to GitHub.

                                            1. 3

                                              This is an awesome thread!

                                              This week, I hope to use my nasty http://github.com/dutc/didyoumean assembly-patching hack to make two recent extensions to Python 3 available to folks to play with.

                                              The first extension is decoupling scope-as-visibility from scope-as-evaluation-context, allowing constant expressions to be evaluated either one scope higher or in the highest-level scope. Since we can probably safely assume module-import can occur at any time before its contents' runtimes, this could allow us to encode certain static checks and static optimisations into our code.

                                              Thus, the following would immediately raise a ValueError (day is out of range for month.)

                                              >>> from datetime import date
                                              >>> class Foo:
                                              >>>  def bar(self):
                                              >>>    return !!date(2014, 2, 30) # evaluate at top scope
                                              

                                              Similarly, we could fix a specialisation to a fast path in the below (for the lifetime of the programme):

                                               >>> def f(x, y):
                                               >>>  def helper():
                                               >>>    return !{(int, int): fast_path_ii, (float, float): fast_path_ff}.get((type(x), type(y)), slow_path) # evaluate one scope up
                                              

                                              The second extension adds ast-literals to Python 3.

                                              >>> e = $(x+y)*w+x
                                              >>> e
                                              sym(+, [sym(×, [sym(+, [sym(x), sym(y)]), sym(w)]), sym(x)])
                                              >>> e(x = $w+3) # bind x
                                              sym(+, [sym(×, [sym(+, [sym(+, [sym(w), 3]), sym(y)]), sym(w)]), sym(+, [sym(w), 3])])
                                              >>> e(x = $w+3)(w = 10) # bind x & w
                                              sym(+, [sym(×, [sym(+, [sym(+, [10, 3]), sym(y)]), 10]), sym(+, [10, 3])])
                                              >>> e(x = $w+3)(w=20)(30) # bind x, w, y
                                              sym(+, [sym(×, [sym(+, [sym(+, [20, 3]), 30]), 20]), sym(+, [20, 3])])
                                              >>> e(x = $w+3)(w=20)(30)() # bind x, w, y, & evaluate
                                              1083
                                              

                                              Not sure if either of these are actually useful, but they are fun exercises!

                                              1. 3

                                                Finally getting around to catching up on all the new and juicy goodness of Ruby2.0 and some old and juicier things that slipped into 1.9 something when I wasn’t looking. (I learnt Ruby on 1.8.6)

                                                In particular Enumerator.

                                                Good links found so far…

                                                https://practicingruby.com/articles/building-enumerable-and-enumerator

                                                http://blog.arkency.com/2014/01/ruby-to-enum-for-enumerator/

                                                …and this is why I became motivated to start writing in that style…

                                                http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321

                                                1. 1

                                                  At work I am mainly being smug due to my team’s system (which used to be blamed for any problems suffered in any other system) was the only system that didn’t fail in the onslaught of sales traffic last week.

                                                  At home I am still tinkering with Clojure & Datomic.

                                                  1. 1

                                                    One of our products is an Enterprise Social Network 1 ( think “Facebook for the Enterprise” or “G+ for the Enterprise”), and right now I’m doing a little reworking of the “update status” input box. I’m hiding the “post” button by default and only revealing it when the box has focus.. and also revealing a new mini-toolbar which has the controls for doing things like attaching a file to the post. I’ll be moving from that to actually adding the file upload support, and figuring out what to DO with files…. we’ll probably offer the ability to put files into a “thing” where the “thing” is Google Docs, Alfresco, Sharepoint or another DMS… but we’ll probably also have a lightweight approach to storing files for people who don’t have another DMS to work with. We might embed Apache Jackrabbit or something, but that’s all still kinda up in the air.

                                                    We’re also planning a “Spaces” feature (something like “Team Rooms” in Lotus Notes or whatever) for groups to have dedicated collaboration capabilities. I’ve been reading and re-reading a lot of Douglas Engelbart’s work on “Dynamic Knowledge Repositories” and thinking about how to integrate some of his ideas into our approach.

                                                    Other than that…. hmmm… well, I’m (ghost) writing another Infoworld article. And tomorrow I go to the cardiac rehab clinic for a “stress test” as I begin cardiac rehab to recover from a heart attack.