It’s Monday, so it is time for our weekly “What are you working on?” thread. Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators?
It’s Monday, so it is time for our weekly “What are you working on?” thread. Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators?
Hello all and welcome to my weekly update!
I had a change of pace in the last week, and got to go on a short trip to Koh Samui in southern Thailand for a long weekend. The views there are incredible. I managed to keep up my GitHub streak though!
In Open Dylan land, I have submitted a pull request that brings back
dfmc-execution, an interpreter for the compiler IR. This is some experimental work and has known issues, but could be interesting down the road. I also improved the documentation on the name mangling used by the compiler.There is an HTTP server written in Dylan. I removed the broken support for HTTP/0.9 and started looking at an optimization to how we buffer data (by avoiding the streams framework) and working more directly with buffers.
For debugging support, I have been digging through a lot of the internals of LLDB and thinking about how to improve things. I also have a branch that adds experimental support for the Dylan runtime as described in the update last week. Some significant work needs to be done to improve the ability of LLDB to support additional languages that do not use clang. If there are people or organizations interested in funding or supporting this work, I would be interested in hearing about that. This work would benefit Rust, Julia and other languages.
In terms of improving the type system in Dylan, one of the first things that needs to be done is to add support for function types and teach the compiler how to make use of them. Many other improvements build on the same infrastructure required for that, so it seems like a good place to start. I am struggling to find the time to take this beyond the planning stages though, so I would welcome help from anyone that might be interested in working on such a thing.
I have also submitted a number of patches to emscripten to upstream changes from a client for my contracting work. I have more on the way, but am pretty close to closing out a major aspect of the project that I have been engaged with for a while.
For the upcoming project that I am looking to launch, it is an open journal on a highly technical set of topics that I am very interested in. I have gotten suggestions that it be peer reviewed and perhaps be more formal with DOIs and set up with permanent URLs for versions of documents to make it easy to cite. I am looking at perhaps using software like OJS or rolling my own static publishing system to power the site. Does anyone have experience in this area and be interested in talking?
Until next week!
Making some improvements to a book I wrote on web dev in Haskell (new chapters, reorganizing, add'l content)
Building out a couple side projects, which means debugging some libs (hit a postgres error the other day) and writing some integration/scraping stuff for things that don’t have apis (creating structured content from disparate products). Also maybe writing another haxl lib and pushing a new release for an old one.
I want to play with integrating purescript into my gulp build process, but likely won’t have time for that.
$work: SAML, SAML everywhere.
We’re trying to do some ZenDesk integration stuff, I’ve been tasked to set up a PoC Identity provider to see if we can actually do it. I am not particularly convinced it’s worthwhile, and reading about this stuff makes my eyes bleed, confirming my theory.
!$work:
Probably nothing, really. At least nothing technical. SAML has ruined me for writing code for the time being. I’m dealing with a bit of a infestation of what I think are fruit flies on my zinnias. I’ve found that a bit solution of 4 drops of dish soap (not antibacterial) and ~32oz of water (half a gallon), shaken thoroughly, and sprayed across the plant is enough to dissuade the bastards, but I am a bit worried about the effects of the soap. Everything seems to be okay, and the infestation is somewhat reduced, and there’s no noticeable change to the leaves. They should be flowering in a couple weeks, so hopefully they’ll be okay. I’ve also been planning on getting some 40K models, to add to my list of nontechnical hobbies, I haven’t played since I was a kid, and I only ever used a buddie’s older brother’s models, so I’m excited to actually get to try painting and playing stuff I want to play (though I do like the Orks, still, which is what I played back them… something something git ‘em boyz.)
SAML? My condolences … that was the first time I was happy to throw up my hands professionally and beg for a consultant.
I’m working on an academic implementation of the RAFT protocol for a system to use in a Monte Carlo simulation for my math course. It’s proven quite fun and thought provoking, though I’m still working my way through it! Oh and I’m writing it in Clojure, which is most curious to me as I have only dabbled in Clojure and FP so this project has been very educational.
Doing some FPGA consulting work on the side. Currently investigating how to perform DMA between the HPS memory and FPGA DDR3 SDRAM on the Cyclone V.
I’ve sort of run out of steam on my personal technical projects after “completing” the Creek vector processor. Since I only have a month until starting grad school, I’ve decided to take it easy on that front.
On the nontechnical side of things, I’m trying to finish reading Water Margin, the classic Chinese novel about Song dynasty outlaws. I checked out the Pearl S. Buck English translation from the library 7 weeks ago and I’m almost done. It’s a very long book.
Work:
Refactoring our Clojure codebase to use…something sane. Currently still researching, but Prismatic Graph looks like the best fit so far.
Not work:
Swearing violently at CSS for my personal site (featuring tsm-the-musician, not tsm-the-programmer). I’m growing increasingly unsatisfied with the Octopress theme I have, so I should probably just nuke it and find a new one.
At work building out a test framework, which is slowly coming along.
Not-work is a bit of a lull. Last worked on bringing a text-adventure-by-sms project on-line and taking a bit of a break. Started reading Let over Lambda and had to backtrack and read On Lisp first due in part to the implicit assumption that On Lisp was well known to the reader.
I’ve been reading On Lisp (again) on the bus to tackle LoL next. It’s slow reading for me, though – read through some macros, stop and think about them for a bit.
I quite like On Lisp. I’d read a lot of stuff that had talked about how powerful designing programs in lisp is, but it’s nice to get concrete examples of useful abstractions.
I’ve been hacking together a “virtualized GSM phone” (this game really does not have a name) using a Huawei 3G modem, a Raspberry Pi and some Scala code. I’m scratching an itch with this project, namely that paying for roaming is silly.
Late to the party! Ive been working on translating SDL from C into Go so you can have a graphics library in Go with minimal dependencies. In the process ive written a helper tool to translate a lot of very common, easy to switch difference between Go and C. My goal this week is to get a lot of the unit tests finished so I can then focus on “turning green to red”
[Comment removed by author]
We did the same thing last year except instead of Spark, it was Storm.
This week I tried to get a streak going by working on Open Dylan’s libgit2 bindings.
The heavy lifting is done by Melange, but because of the differences between C and Dylan, e.g. multiple return values, I have to tell it how to change the generated Dylan functions to be more Dylan-ish.
Other changes include making the interface easier to use by using named parameters (keyword parameters in Dylan) and sane default values.
My goal is to have the library in an usable state by the end of the week, then polish it and complete the missing bindings. For this reason I’m working through the 101 Libgit2 Samples, both as to provide basic test coverage and as a way to learn how to use libgit2.
While working on this I often noticed a crash occurring when returning a struct by value, it turns out it’s a known bug in Open Dylan’s C back-end. When I’m done with the bindings I may tackle this problem because it looks interesting and allows us to move forward on the Obj-C bridge front. And since in October I’ll be moving to another country (from Italy to London, UK) this means I’m going to use my MBP only (I’m not going to take the linux box with me) and I want to play with the Obj-C bridge a bit.
Going forward with my most personal project (the website is done with a pressKit-like thing I made with Polymer last week), made cutscenes today and still making a map. Hope to get a decent demo as soon as possible..
Extra: While making the cutscenes I stumbled upon this error, who writes those?!
Spending my evenings writing a very custom purposed XML “schema” inference engine. The generated schema is used to generate strongly typed C# classes and serialization. Currently the project is using LINQ-to-XSD to generate the classes and .NET’s built-in XmlSchemaInference class to do the XSD creation but the whole process has many flaws I’m hoping to solve. Mostly due to XSD being a poor schema format for our needs.
Working on some keyboard input bug in gtk+ on Win32. Sorry it’s not the most exciting thing to report.
Monitoring some upstream .deb releases and loading them in to an apt repository for easier installation.
Two things in TokuMX:
Just finished the tutorial section of my series of blog posts “Mutable Algorithms in Immutable Languages”. It’s easily the most ambitious blogging experience I’ve tried so far and it’s been a lot of fun.
I’m going to gather up what I’ve learned through the process and write the “epilogue” post next where I try to examine what’s going on a little more theoretically and a little more formally.
I miss the bot.
I have been reluctant to say what I am working on because I thought it might start PMng me for status updates.
Sigh!
Unsexy refactoring to add yet another phase to our build system….
Self tests for tools.
Interesting approach this time.
Using mercurial patch queues and http://mikadomethod.org/
The idea being I should be able to do a “push” to mainline at least once a day.
At work, trying to get up to speed with OpenStack bare-metal provisioning, and keeping up with our OpenStack installer project.
At home, probably going to keep working on my first simple Racket-based web site, and might hack a bit more on my git-annex photo importer - it needs to get a lot faster.
Or I might just try to get enough sleep this week.
Finally starting to write some tests for the web app that I’m working on. As a newbie to web app testing, does anyone have recommendations on how I should be approaching this? Thought processes/paradigms? I’m working with Python/Django.
At work: getting some late breaking features in before our next beta. Nothing too crazy, just lots of polishing before hopefully shipping next month.
At home: I got nerd sniped this past weekend. (I can’t resist a good graphics challenge.) Now that that’s done, I’ll be getting back to learning Qt and working on my little map tile art editor for games.
I am spending some time on awesome-scala list, which gained traction few days ago - https://github.com/lauris/awesome-scala
So far the community has been awesome and we are improving the list constantly.
Waiting for, receiving, and then setting up my new MacBook Air in anticipation of an upcoming hackathon.
Last week I added audio support for Fire★
This week I added an audio encoder that uses opus codec. So you can take the PCM audio, massage it, encode it, send it, decode it, and play it.
The biggest flaw is that currently have no easy way to combine many audio streams into one. So unfortunately voice chat only works well with 2 peers.
The rest of the week and following week I am going camping. So fixing this flaw will have to wait.
Unless of course someone here wants to do it, hint hint. code is GPLv3 and is on github.
Working through 4clojure and taking this as an opportunity to test out the Light Table editor. The “instarepl” feature is helpful for getting my bearings in the language. However, I’ve missed vim keybindings for manipulating text.
Also, I’ve been reading Getting Things Done. I’m not usually one for productivity books, but a coworker suggested it.
There is a plugin for Light Table that provides Vim keybindings. (Likewise, another one for Emacs.)
Last week, wrote a big article about Apache Storm called “Apache Storm: The Big Reference”:
http://bit.ly/storm-books
In open source news, my team merged Python 3 support into streamparse, our Python + Storm interop layer – that’s pretty great. It was also our first significant contribution from the community.
I also developed a proof-of-concept of using ElasticSearch with Storm to index about 20,000 documents per second to implement a real-time indexing layer for my time series database. It worked like a charm. I took advantage of ElasticSearch’s alternative storage and loosened durability options, such as mmapfs storage and refresh_interval tweaking. Really loving ElasticSearch lately!
This week, I’m probably going to do more work with Cassandra, ElasticSearch, and Storm, moving a new analytics backend from staging to production.
Hello, I’m new here. Waves.
I’m working on an encoding API service to compete with Zencoder (now Brightcove), Encoding.com, HeyWatch, PandaStream, Transloadit, Amazon Transcoding, and any others I find along the way. If you know of any good API services that handle encoding of any kind let me know.
Tech I’m using right this moment: Scala/Akka, Play-Framework, CouchDB (I’m still vetting this choice), PostgreSQL, and REDIS. On the front-end I’m using Swift on iOS for a prototype client.
Just a little background - This is my second time writing this type of thing. I did something similar last year for a company called Brabble using Node.js and MongoDB and it worked well. But I can do better. I do have a few things I can’t reveal yet but I can say that it involve GPUs, OpenCL, and some fancy footwork for parallel processing. I hope to be the highest performing service for a reasonable cost.
And if it all fails I’ll probably open source the code in hopes that people can use it in their infrastructure without too much hassle or at least learn from my mistakes.
Work: Doing presentation/poster stuff through tomorrow (with any luck >.<) and then back to cranking out JavaScript.
Not Work: Prepping an application to the Fullbright program (grant for US students to do research outside the US), learning OCaml.
Work: Working on a C# Website using KnockoutJS, ASP.NET MVC and SQL Server.
Work-Related: Spending all my free time studying/finishing up a Clojure presentation I will be giving to co-workers on Thursday.
Looking into upgrading my pipes-csv Haskell library to use the new lens-based pipes-parse idioms. Other than that I’ve been attempting to build a Datomic-like database in Haskell.
Reading some things on neural networks, since one of my friends got me excited about them. I’m hoping to make an actually usable version of a hack I made two years ago for decrapifying cellphone photos of handwritten notes.
If you want to help me get diversity in my training set and you have some handwritten notes, you could email me some photos of them :)
I’m probably too late to post on this, but here it goes anyway.
For work I’m writing a distributed stream joiner, to join 2 or more streams of sensor data based on the contents of the messages and output results as they arrive. I previously thought I had to use a relational database or write my own data store, but over the weekend I realized how I could use Cassandra to do it so I’ve been working constantly on it ever since.
I also have to prepare a workshop for Cloud Identity Summit this weekend in Monterey, CA. This comes at an unfortunate time when I can’t think about anything other than joining streams of data. Normally I think I’d be excited about it, but not this time.