1. 12

This is the weekly thread to discuss what you have done recently and are working on this week.

Please be descriptive and don’t hesitate to ask for help, advice or other guidance.


  2. 9

    I’m in the second week of a new job! After a nine-month stint as a manager at Knewton, and a longer stint as a kind of evangelist-blogger-coder-mentor-hybrid…thing at Fog Creek, I’m back to being a developer full time at Khan Academy. And I don’t know entirely what I’ll be doing specifically; right now, I’ve been dusting off my JS knowledge, diving into the website, and maybe today trying to learn enough Android or iOS development to knock off a minor feature or two there. It’s exciting to be relearning everything after awhile off.

    1. 3

      Congrats on getting back into hacking full-time. I did that about 1.5 years ago after being a PM and it’s super fun, you’ll enjoy it even more than you used to. Also Khan Academy is totally awesome!!!

      1. 3

        update your lobsters “about” blurb :P I’ve gotten the impression through various channels that KA is a wonderful place

        1. 1

          Heh, updated it. Thanks for the reminder. And so far it definitely is wonderful.

      2. 8

        I’ve been continuing to work on Helix Pi, my game development tool for making games without coding.

        As a user, you describe with animations how your game behaves, and then Helix Pi uses genetic programming to create the code for your game for you.

        I’ve been working on it for 3 months or so now and I’m getting close to being able to create Pong. Recently I’ve been working on helping programs learn multiple behaviours. That works by lending more credence to programs that aren’t very good on average, but excel at something the rest of the population is bad at. This improves diversity, and the branch was cheekily named affirmative-action.

        In the process of implementing that feature, I regressed the fitness of my baseline test significantly. Simply put, Helix Pi is now more stupid. That’s a bit annoying, and it’s not fixed by simply disabling the feature. That was a valuable lesson, because if I had committed in smaller chunks and avoided breaking my core test, I could have bisected back and found the regression. Now I’ll have to hunt it down.

        Next up on my feature list is a roulette wheel breeding system. I’m currently using truncation breeding, effectively taking the top half of the population. This is pretty awful from a diversity standpoint, so I think I can make large improvements in this area.

        If you want to try Helix Pi out, you can play with it in your browser here. I think it’s broken in Chrome right now, sorry about that.

        1. 2

          This sounds a little bit like Scratch. Just a bit. What do you think of Scratch? It’s a programming language, but particularly well-suited for kids and simple games.

          1. 4

            I gave a talk about Helix Pi recently and I cover that really well in the first couple of minutes.

            For those who don’t want to watch a video, the line I draw is that visual languages like Scratch require a lot of the same skills as programming. In order to use Scratch to make games, you still have to be able to break down the behaviors of your game into exact logic, that you then implement yourself. If you don’t arrange your blocks perfectly, your game won’t work.

            With Helix Pi, you create animations that describe behaviors of the game, and the code is evolved to perform those behaviors. You don’t need to understand the logic of how your game works at a computer level, you just need to understand what you want to happen in response to input and time.

            One of the things I’m consider is allowing users to directly modify the generated Helix Pi code in a Scratch style interface. This would mean you could evolve the bulk of your code, modify it and have those modifications carry on in the evolutionary process.

            So I actually view Helix Pi as a higher level language than Scratch (probably one of the highest level languages so far).

            1. 2

              I’ve played a lot with Scratch - but don’t like the new dependency on Flash - so have been playing with Snap! which is a web version of scratch that grew out of Berkeley’s Build Your Own Blocks project for Scratch.

              1. 3

                The flash thing is sad. Is it really impossible to finish the HTML5 implementation of Scratch?

          2. 8

            Week 2 of the new job!

            This is my chance to really learn Django. I need to improve some models at work that are making it difficult to organise our data and are making some queries really slow. I’ve been reading the pleasant 2 Scoops of Django to get a feel for what the habits and customs are around these strange new lands.

            Outside of work, I have an 11-year-old niece visiting us from out of town. It’s a lot of fun to take her around Mtl and show her everything. Having a kid around is a great excuse to run around in Parc Lafontaine’s playground and swing around the high-flying pirate obstacle course.

            1. 6

              I am fighting with Qt on my Jetson. From what I can only imagine as malevolence, the board runs Ubuntu, and getting Qt to play nice with gstreamer is deeply unobvious. And unpleasant. nVidia supplies a nice hardware H.264 decoder, but gstreamer has been ignoring it in favor of … nothing at all. And YHWH help you if you try to get around this problem with apt.

              So this week is either getting Qt to reliably play back BOG STANDARD H.264 or finding some other framework to build X applications with on the Jetson. Which … argh. I’ve written H.264 decoders before, but a) I’m not that interested in doing it again and b) time is constrained and most importantly c) there’s a perfectly good hardware accelerated one already installed.


              1. 5

                I’m working on a version of netmap-ipfw with a bulit-in JIT compiler (which uses LLVM’s libraries). So far, I managed to get it working with LLVM 3.5 (kind of old, but I had to port it from what was last year’s llvm-devel to today’s standards, without using too much time).

                I’ve found a bug in netmap that makes packet filtering something impossible. It worked last summer with a low number of packets, but it seems everything falls apart now when trying to filter anything.

                Since all the rules are executed by “calling” inlined functions in LLVM’s IR, there are some issues with any change in the control flow. That’s why I’ll also have to implement by hand with IR a pairs of commands (skipto, and call/return).

                Just FTR, this is a Google Summer of Code for FreeBSD.

                1. 5

                  Still working on the bootstrapping phase of my language, Kropaya.

                  It is written in Emacs Lisp, with no extensions or dependencies, as Emacs Lisp is a long term stable language that is available on basically any developer’s system. Not using any dependencies means that I have to write things like my own parser combinator library, though, which is slower going than it perhaps should be.

                  Code is here: https://github.com/flaviusb/kropaya

                  1. 5

                    This week I’m focusing my spare time efforts into 2 things:

                    1. Learning to surf: It’s been a long time mission of mine to learn to surf. While I can surf now, I’m not very good at it. Now that I finally live a 5 minute drive from the Pacific Ocean (in SoCal), this needs to change.

                    2. Implementing CFDG in Racket. I’ll eventually have enough free time to make a language (with syntax) for it, but for now, just focusing on a DSL and runtime to support it.

                    1. 4

                      Lots of stuff… Over the past 9 months or so, I haven’t gotten a lot done, as I had a heart attack about 9 months ago, and - TBH - as I’ve been recovering, my priorities had been elsewhere for a while. For the past while, I’ve been more focused on bicycle riding and trying to stay active, then coding and working on the startup. It’s really only been in the last week or two that I’ve started getting back in my groove a little bit.

                      So, what am I working on? Well, one of our products is an Open Source Enterprise Social Network1, and lately I’ve been working on integrating Apache OpenMeetings2 so we can offer seamless, built-in video conferencing / chat / whiteboarding around the entries that show up in your feed. What makes that doubly interesting is that our ESN is based around deep integration with business activities, so instead of a feed full of pithy status updates from colleagues, and stuff like “who wants to go have Thai for lunch”, you can subscribe to business events, tasks from a BPM server, calendar events, notifications from other applications using ActivityStrea.ms, etc. IOW, the kinds of things that actually (sometimes) require some action, and where you may want to have a discussion with colleagues before taking action.

                      I’m also working on some marketing materials, as I try to refine our messaging. A big part of our emphasis is using Semantic Web technologies in the enterprise to support knowledge management and collaboration, and we’ve been working on establishing the phrase “Enterprise Knowledge Network” to represent what we’re promoting. So working on some marketing collateral, trying to figure out how that fits in with some older messaging we have up on the website now, etc.

                      On that note, if anybody would like to read over this document and give me some feedback, it would be MUCH appreciated.


                      Don’t be afraid to be harsh, this is like Draft One, so I know it has issues. :-)

                      1. 4


                        • Last week was the semi-annual company hackathon, where Alan Malloy and I worked on making Clojure use typed local variables instead of Object typed locals. This reduces average method size by making many checkcast operations uncalled for and thus gives the JIT more rope to play with in optimizing hotpaths. We saw ~10% or more speedups. Today I’m working on finishing up those patches and getting a blog post on the matter out the door. Edit: Link
                        • End of summer presentation prep T_T will also come with a blog post about using Make with Clojure (yes really)


                        • Dota 2 International!
                        • Still stewing on Ox in the backgroud. Pondering scope and taking a JavaScript first approach so that I can more reasonably experiment with an integrated “brilliant” IDE as I develop the language and what I really hope to get out of this project.
                        • Reviewing my GSoC student’s work on packaging source metadata about Clojure artifacts.
                        • Bracing myself for the end of my internship, returning to Texas and resuming school
                        1. 1

                          All I have to say about Ox is … 好牛啊! A fast statically typed lisp is an interesting idea. Can’t wait to see how this turns out.

                          1. 3

                            You might be interested in Shen!

                            It’s statically typed – think Haskell meets Lisp, with a little Prolog for good measure. Its core is a very tiny and simple lisp called K-Lambda, which allows it to be a very portable language. The canonical implementation is written in Common Lisp, but there are implementations written in Java, Python, Ruby, Javascript, and so on.

                            Google Group here

                            1. 1

                              Thanks! Fair warning, as the blog posts state this is my slow burn one day it may work vaporware side project so don’t go betting on a delivery date :P

                              The really sticky bit which I’m still trying to wrap my head around is how much of an ML/Haskell type system can you profitably integrate with a Lisp take on the world. Also the fundamental problems of compatibility and utility remain…

                          2. 4

                            I started on a website but only recently learned about mvc and other frameworks. So I’m going to be redoing my site. But mvc is so hard…

                            1. 2

                              Get a good Rails or Django book (or similar), type out all the examples by hand, and don’t stop moving forward each day until you’re done. You’ll understand MVC soon if you just stick with it. I find books are the best way to learn because they’re comprehensive and don’t make too many assumptions about what you already know. Keep at it!

                              1. 1

                                You’re right. I only started learning to code less than a year ago. I can do html and css and enough php to get started. Thing is I got arrogant and thought coding was a piece of cake. Til I heard about frameworks and took a peek at lobsters code. It was humbling. Now I’m going to continue with the books. Thanks!

                            2. 3

                              We surpassed our goals last week and have most basic functionality working in the Peergos demo - we don’t have an ssl cert yet so ignore the certificate warning.

                              Next up is the social layer: following, sharing etc. We would love any feedback on the demo as it stands.

                              Peergos is an end-to-end encrypted, peer-to-peer file storage and sharing and communication network.

                              1. 3

                                I’m working on implementing some basic load balancing and retry logic from our node.js server when communicating with clusters of backend services via HTTP. I’m usually a Java dev so a lot of the node patters are new to me. I got some great direction from the Joyent documentation on Error Handling in javascript/node.js. A really great piece of documentation that clarified a lot of stuff for me around sync/async error handling.

                                I wasn’t able to find any existing libraries that handle these kinds of problems. If anyone knows of anything they could point me to that would be great!

                                1. 3

                                  I’m still working on rsedis, a Redis re-implementation in Rust. Last week I was able to run some of Redis’s tests, and I’m gradually enabling them, which is helping me find bugs in my implementation, corner cases that were not handled, and more importantly discovering undocumented features in Redis, and hence contributing back to redis-doc.

                                  1. 2

                                    Now that RustCamp is over, I have some breathing room. Whew!

                                    1. Reply to my massive backlog of email
                                    2. even moar Rust docs, as always. Haven’t decided what specifically I’m focusing on this week.
                                    3. Hopefully ship the MVP of Artisan Assistant to replace the current prototype which is still in production.
                                    1. 2

                                      I’m trying to figure out if there is some deterministic algorithm for this problem. Initially I thought this would be known, but seems like it is not as trivial.

                                      1. 2

                                        Starting planning for a new API that’ll provide content for syndication and mobile app. Idea is to prove the notion with some simple endpoints extending basic work we’ve already done using ngx_lua/OpenResty in front of Neo4J, with a proxy_cache setup incorporated to serve flat responses when possible & take some heat off Neo. So, bit of investigation today into Kong for scaling out the API longer-term, & doing OAuth provision; seems decent after a basic install & poke around. Also rooting around in Backbone to see whether we could maybe auto-generate some of the model code from the Neo REST API’s traversal stuff. Dunno if that’ll be possible yet but v interesting to investigate, and would be really nice not to have to futz with API signatures by hand too much over time. Really starting to dig OpenResty, lua is cool (and fast, with JIT), and exposing the nginx process/phase internals to lua means you can control what’s going on at quite a deep level without having to get into C.

                                        1. 2

                                          Work: more ontology matching and plumbing, at least once updates to the mobile client build / deploy pipeline are done.

                                          Elsewhere: a few more heka + lpeg log parsers for the Artifex server.

                                          1. 2

                                            Ooh! Since I’m on vacation, and using it to work on my open-source project, I can actually answer this for once. :)

                                            I’m currently going through the issue list of major concepts that need to be taken out of my thing*, and I’ve indeed removed the big ones. The ability to refer to any previously-given term based on its structural hash is gone, which means terms are now referentially transparent, with lexical scoping being the only kind of scope. It was surprisingly difficult to remove that assumption; it means there will eventually need to be a different mechanism for accessing pre-defined terms that are part of the standard library.

                                            At some point I need to read through my backlog of papers on type theory and see whether any of them have advice on what to do about type reflection violating parametricity, but I’m going to put that off at least until the evaluator and type checker are implemented at all; it’s embarrassing how long this project has existed without those. (Though, in fairness to myself, I had several major mental-health events that brought progress to a halt for a couple years.)

                                            My immediate hope for the next two days is to implement serialization and deserialization of the “explicatory”, text-based format. When that’s done, it’ll be possible to write test cases for the evaluator, and start implementing it. It’s unlikely that the type checker will happen this week, so I’m trying not to hope for that. I do have a much better understanding of how the type checker will work than I did when I started, at least.

                                            *“A self-describing binary data format for dependently-typed object graphs.” You can see why this is a footnote.

                                            1. 1

                                              How is a binary data format self-describing? Could you elaborate on that?

                                              1. 1

                                                What makes XML or JSON self-describing? Or ASN.1, or protocol buffers, or flatbuffers? The same thing. :)

                                            2. 1

                                              Attempting to rebuild angularjs from scratch.