1. 22

It’s Monday, which means it’s time for our (semi-) weekly “What are you working on” thread! Please share links and tell us about your current project. Do you need feedback, proofreading, collaborators?

  1. 11

    Hacker News picked up my printf Idris video and one asked “does this only work for constant strings or can it work for runtime values?” which is something I’ve been asked quite a few times. I want to make a video to follow up on the question.

    Fundamentally it comes down to: how do we make code like the following work?

    main = do
      userInput <- getLine
      printf userInput "Hello" 2

    Idris doesn’t allow throwing exceptions, so we can’t make printf just throw when the user inputs something without “%s” and “%d” (in that order) so we have to provide evidence when they did but safely exit if they didn’t. You end up doing something like this:


    Which is just checking the user value and building up the proof which Idris would have otherwise done with a constant value. So it’s just validation, but building a value to be provided at the type level.

    1. 8

      I mentioned a few weeks ago I was grinding through polishing a Ruby gem to support the new two-factor authentication standard. I have finally soft-launched that and it’s on GitHub here. If you are a Ruby coder and could tell me what you think of the readme/gem so far I’d appreciate the feedback. If a truly brave soul would like to experiment with putting it into a new/existing Rails app I would hugely appreciate hearing about that experience.

      Today I am finishing the install instructions and the homepage (which is already linked from the README…). And also realizing some significant technical changes I’d like to make (integrate auth into username/password forms rather than as separate step), but right now it’s more important it get out the door than be perfect. Rest of the week is dealing with anything that comes up as people start to see the gem.

      Happy to talk 2FA/Ruby/oh god finally finishing, but I’ll do a round of “Show Lobste.rs/Show HN/etc” posts, most likely later today. If you want to talk about the everyone-has-a-strong-opinion topic that comes up at the end of TwoFactorAuth’s readme, please hold it until then. :)

      1. 1

        Great to see a FIDO/U2F lib getting out there. Great work, and may you get useful feedback on it.

      2. 6

        I’ve already done a lot of work today, despite being somewhat sick. In fact, I just finished up everything I wanted to do last week. :)

        There was a bit of a grind involved: I needed to decouple the runtime representation of Python objects from the parser representation. This required me to rewrite evalExpr to stop leaning on Constants, which I was using to pattern match unary and binary operators. The revised version is a bit more succinct.

        After that, I renamed my Scope record to be Env, to better match Python’s nomenclature. An Env is essentially a list of places to do name lookup in. That was a bit painful, but the compiler is always happy to tell me where I’m wrong!

        Finally, I had all the pieces in play to add the notion of function bindings. This allows me to call functions in another module, and have their name lookups succeed (needed for recursive calls, or if they reference globals). I stored an Env on the Function itself, and used it during invocation time and it Just Worked. Always a good feeling when that happens!

        This week? I’m not sure. I’ll be finishing up imports soon I hope.

        1. 4

          Should write another blog post about the evolution of your interpreter from “The simplest possible language” to now.

          1. 2

            A post along these lines would be wonderful as I’ve seen people do similarly with their personally augmented and modernized versions of the “Scheme in 48 hours” tutorial interpreter. Nobody’s really documented what they did though.

        2. 5

          Haskell contract work continues. Just ran into my first design decision where Aeson leaning on typeclasses has made me uncomfortable with my options. Record of functions would be better as it’s not clear the same data type will get rendered into JSON the same way by different API endpoints. Typeclass canonicity just isn’t really what I want here and orphan instances will force me to take a second shower. Still really liking Yesod so far, about to clean up some obvious Maybe monad code and figure out how to make it play nice with HandlerT.

          Working on the book. Started kicking around material salvaged from my monad transformers talk by my coauthor. Resuming talks with a couple of publishers soon now that the holidays are going to end soon.

          Rediscovered the joys of trolling Bandcamp for quality black metal.

          I bought slippers. I started crate-training my dog. Talked to an old friend, going to teach him to shoot when he visits Texas next. I’m using the Leerburg materials on dog training to guide how I train Papu. He stays on a leash in my apartment whenever he’s not in his crate. Contemplating leaving Austin for somewhere more remote in Texas after my lease runs out next summer. Joey Hess is my haskeller / lifestyle hero.

          Edit: I wrote a simple MyDatatype -> Value function. Quite satisfied. I could’ve also just filtered/manipulated the Value returned by a canonical toJSON.

          1. 1

            Why wouldn’t a newtype have worked?

            I’ve been thinking about moving to the middle of nowhere (but in Australia) and doing Haskell. Hopefully I’ll be able to do that soon.

            1. 1

              Why wouldn’t a newtype have worked?

              It would’ve, had there been a ToJSON instance anywhere at all. I didn’t feel comfortable making an incomplete “view” of the data the canonical ToJSON instance or putting it with the database entities (diff library). My MyType -> Value function was simple and didn’t require my asserting a canonical instance where none existed. My discomfort with asserting canonicity is that it only included 3 of 12+ fields from the domain data. It seemed a case of “just a function”, no typeclass instance needed. My JSON Handlers in Yesod return Value anyway.

              I’ve been thinking about moving to the middle of nowhere (but in Australia) and doing Haskell. Hopefully I’ll be able to do that soon.

              I wish you luck and joy then. I’m definitely done with city-life for awhile after 7 years of NYC, SF, and Austin. I want nice people, nice dogs, nice food, nice books, and to be left alone.

          2. 5

            I’m exploring different mediums for The Secret Lives of Data project. I’m studying up on motion graphics and After Effects to see if that would be easier and more effective than the D3.js approach I did with my Raft visualization. I’m also researching how to do effective voiceovers.

            If anybody has any tips on After Effects or voiceovers I’d love to hear them!

            1. 4

              For voiceovers, I’ve done a few for client project reviews (clients love when they have a nice video with a pleasant voice narrating what they’re paying me for). With the assumption that you’ve never done one before, here is some advice I’ve gleaned from my experience.

              • Get a decent mic and a pop screen.

              You can improvise a pop screen in a pinch with a t-shirt or other piece of thin cloth stretched around a clothes hanger. Just hold it about halfway between you and the mic (you want to be about 3 ft away, give or take). Sit comfortably but not too close, put the mic slightly off-center from you, these things will help reduce puffing/popping, as well as harsh sibilance (the nasty screeching ’s' sound) and overly bass-y tones. Good mic’s can be acquired for relatively cheap. A USB mic might run you 50$, but the quality difference over a webcam mic or (FSM forbid) a headset mic is astounding. Look for something with a bit of a shock-mount to it to help reduce hum. Don’t spend absurd money on anything, I have one of these which I rather like for skype calls and the occasional bit of impromptu guitar recording.

              • Scripts are good, but don’t be afraid to ad lib.

              I usually try to write a general script which covers the main thrust of what I’m talking about, and then vamp a little to explain detail as I see it. Some people like to write everything out and just read the script, but this has a tendency to sound very – well… scripted. Unless you’re very good at both writing natural speaking prose, and then reading that speaking prose in a natural way, leave yourself a bit of room to improvise.

              • Record everything, even the screw ups, and do lots of takes

              Editing programs – even the somewhat featureless stuff like Audacity – make it very easy to duck in audio and generally edit together multiple takes. If you just get in the habit of recording absolutely everything, you’ll find it very easy to pick out the good versions of sentences from across half a dozen or a dozen takes. I’ve been known to duck the occasional word.

              • Effects aren’t just for music.

              A little reverb here, some compression there, a bit of EQ, sooner or latter you have a really nice vocal. Even if you just use the ‘small room’ preset on whatever reverb effect you can fine, and set the dry/wet to around 10%, you’ll notice a marked improvement in how ‘real’ the vocal sounds. A little goes a long way, and you can really make an okay, obviously dubbed voiceover turn into a lovely, seamless track. If you want some more specific advice, I’m happy to share what I know, but I’m definitely an amateur.

              • Listen through a bunch of different sources.

              Listen on good headphones, on bad headphones, on speakers, on laptop speakers, etc. Listen on a variety of potential sources, try to make it sound good on most of them, but don’t worry if traditionally shitty sources (laptop speakers, crappy/cheap headphones) sound worse than traditionally better or good sources (decent speakers, good headphones, etc).

              That’s about all I’ve got. Good luck!

              1. 1

                Wow! That was a lot of great advice. I’ve never done voiceovers before and I haven’t found many good resources on voiceovers or voice acting in general. I’ll definitely hit you up if I have more questions!

                1. 1

                  Sure thing, tbh, I’d just look at resources for vocal processing in songs. Most of the same principles will apply. It’s actually a bit easier in some ways, since you don’t have to worry as much about creating sonic ‘space’ for the vocal to sit (unless you’re going to have some music in the background).

            2. 4

              This week I’m working remotely while still visiting family in Denver. It’ll mostly be polishing up documentation and specs on a project I’m working on for work so that I can bring other people on board to help with it next quarter.

              On my own projects, I finally got around to mentioning a project I built to solve my own problems: some command-line crypto tools for storing passwords and one-time password (TOTP/HOTP) tokens and journaling/note-taking. My use case comes down to trying to avoid using my browser and phone at all costs, because I have a strong dislike for using either platform for security-sensitive work (or really using them at all).

              Still tinkering about with my embedded projects, but being on vacation hasn’t been so amenable to getting a lot done on them.

              1. 1

                Very cool Kyle – I’ve been looking for a good cli tool for OTP and password storage.

                1. 1

                  Thanks! If you end up using it I’d be keen to hear what you think of the docs and where they fail.

              2. 4

                One silly thing that Fire★ lacks is protocol and client versioning. I am thinking of adding it this week. What this really shows is my inexperience designing protocols. But damn, it is fun to learn, isn’t it?

                1. 3

                  At work, I’ll be spending some time getting a new service up and running using Postgres and Haskell, which has so far been a joy, using packages like Groundhog (database + migrations) and Servant (strongly-typed web services with documentation generation).

                  For !work, I plan to spend some time working on PureScript stuff, namely reimplementing the Pursuit search engine as a Haskell web service with a React front-end, and integrating some recent contributor work with Bower on our community server. If there’s time, I have a showcase project in the works for Haskell/PureScript, so I’ll be spending time on that.

                  1. 2

                    Thoroughly enjoying the rest of the 31c3, getting my rust rocksdb wrapper in a better state, and then adding some more timeseries instrumentation to cockroachdb.

                    1. 2

                      I’ve been futzing about with my fancy new DK2 Oculus Rift. In particular, I’ve been playing some demos and messing around with the OR support in the UE4 engine.

                      Well, actually I’ve just been playing a lot of Elite: Dangerous and pretending it counted as ‘research’.

                      Christmas and time off has totally screwed my schedule of productive work, but it did leave me with a few ideas about how to improve my development-y life. Specifically in terms of using docker more heavily to get a meet-in-the-middle point between the convenience of developing on linux, while not having to run a whole separate VM (since I have to run Windows for work). I’m hoping to get to the point where I can have windows just act as an interface to a very minimal cygwin/similar shell w/ boot2docker. Various projects would have a dockerfile to ‘compile’ it to a docker image I can then run and muck around in. My hope is that I’ll be able to set up a comfortable environment that’s at least par with my OSX machine, I’ve tried doing this with full VMs before but it’s always been very frustrating. I’m hoping Docker, which has fast become my new favorite way to reason about devops, will help alleviate some of these frustrations.

                      1. 2

                        At work attempting to track down the source of a periodic (On the hour, every hour) latency spike in our Solr cluster hosted on AWS ElasticBeanstalk. We can’t find anything in the logs. Very perplexing and frustrating. Asked AWS support, and waiting for them to report back. (They’ve broken their 4-hour SLA tho…) Also implementing a serve-stale-while-revalidate cache solution for a very hot cache that (we theorise) is badly affecting our response times in the high percentiles. 50% is ~50ms, but worst case regenerating that cached value could be a second or more. That’s a big outlier. I’ve made a patch, and I hope to be able to performance test it tomorrow.

                        Also: I have been shit at timekeeping (or really good at procrastination) lately so have read up about the pomodoro technique over Christmas. First day of trying it today at work & had a wildly productive day. Don’t know if it’s because:

                        1. The technique is working
                        2. I’m rested after a 4 day break
                        3. I’m not getting interrupted every 2 minutes because all my colleagues are still on holiday
                        4. All of the above

                        Anyway, it made for a pleasant change. The challenge is to see if I manage to repeat it tomorrow!

                        At home I’m (very slowly) moving forward in a toy Clojure web development project using Datomic.

                        1. 2
                          • Making minor edits to a fantasy novel I wrote ten years ago. I’ve heard other writers say that they hate reading their old novels, but I actually quite enjoyed reading through mine with fresh eyes.
                          • Writing the next chapter of my Jasmine testing book, then publishing to my readers.
                          • Writing the next issue of A Drip of JavaScript on enumerability.
                          1. 2

                            On vacation till the 5th of January.

                            Hopefully I’ll be skiing a lot, helping my girlfriend’s son with his arduino robot, and maybe working on some lisp projects.

                            1. 2

                              I posted a Show Lobsters about ssh-chat a few weeks back; since then I’ve been working on a full rewrite which decouples the SSH stuff from the chat stuff (feedback on the code appreciated).

                              Btw the original ssh-chat server is still running with more than 50 PRs of improvements merged. Come say hi, chat, idle with fellow Lobsters:

                              $ ssh chat.shazow.net

                              Meanwhile, I’m also drafting up a blog post about all the neat things I learned about SSH while building ssh-chat, and plotting future fun experiments like an SSH MUD or DHT or blog. The silliest most-surprising idea will probably win. (I expect future historians to classify this as my SSH period.)

                              1. 2

                                Finished setting up the microserver with media apps, and have started writing each portion of what I have up into blog posts (a friend wants to know what I’ve done, and it’ll be useful for me to refer to in future too.) Need to put a VPN on the internet-facing zone too, and I think I’m done with tinkering for a while. Pretty impressed with how well the hardware has done what I’ve asked of it, it might be a fairly low spec AMD processor but it still has some grunt.

                                Discovered Exercism.io this morning via someone on twitter and spent an hour doing a few exercises. Reminds me of Codewars which I enjoy doing occasionally, but I like that exercism is using my local toolchain where codewars makes me write code in the browser.

                                1. 1

                                  I’m working on an mobile device analytics. I’ve got 2 reports with all the GUIDs of users who performed specified events. I need to intersect them to find a report of the users who performed both events. But because of the large number of users, simply performing the intersection would be waaaay to slow. And I can’t precompute the reports because there could be thousands of possible events one might wish to analyze in this way. So….I’m using a combination of HyperLogLog and MinHash that I posted about earlier to perform a probabilistic intersection. all this would be pretty straightforward - its just getting the data in and out of HBase in the right form, at the right time is proving tricky.

                                  1. 1

                                    Hopefully getting most of the open issues on the GitHub projects I contribute to closed out for the year end. Vacation has slowed my progress, but I’m hopeful.

                                    1. 1

                                      Still off from work this week, so I’m working on some small side-projects, namely Errio, a Node.js library for serializing and deserialization Error instances.

                                      1. 1

                                        Just finished my tic tac toe implementation in golang – if anyone feels like commenting on the code, I’d appreciate the review:


                                        What could I do better, what did I do that’s weird/not go-ish enough, etc.

                                        Otherwise, catching up on some reading that I’ve had on pause for a bit: SICP, and Clean Code. Will probably start another mini-project in golang to continue learning/improving.

                                        1. 1

                                          I’ve been hacking on an TOTP library in Go over the holidays:


                                          There are a few others out there, but I tried to make it semi-opinionated about all the defaults working with Google Authenticator, making QR-Codes very easy to get, and have reasonable documentation about how to integrate it in a real world application.

                                          1. 1

                                            I am working on getting a fully functional port of my pet language (Myrddin) to Plan 9, or at least 9atom and 9front. It’s nearly there, although there are quite a few FIXMEs; things aren’t as frictionless as I’d like yet.

                                            1. 1

                                              Currently playing around with using an FPGA to build an EPROM/EEPROM emulator which can have its virtual “contents” altered in realtime. This consists of a few basic design blocks: UART, a simple command parser, dual-port SRAM, and a basic synthesis-time configurable mapper to support different EEPROM / EPROM setups (output enable / write enable and various kinds of clocking).

                                              This device is useful for tuning older automotive engine management systems as on these older systems the ECU would constantly read the timing and fuel maps out of the ROM. By emulating said ROM, the maps can be adjusted on the fly.

                                              A variety of commercial devices which do the same thing exist, but most are quite expensive for what they are ($200+) and use antiquated physical dual-port SRAM modules.

                                              I had the project working for one specific kind of EPROM using tons of level shifters and a relatively big Xilinx Spartan6 part with C running on MicroBlaze soft IP for the UART and command processor.

                                              This week I’ve decided to try to make the design smaller and very cheap by implementing the UART and command processor in the design and switching to an extremely cheap SiliconBlue/Lattice ICE40 part. I’m also trying to make the pinout and EEPROM setup configurable to support more host ECUs.

                                              My main challenge right now is that these older devices are 5V and any kind of modern programmable logic only supports 3.3V I/O. This wouldn’t be too bad (just use basic off the shelf level shifting), except that I now want the pinout to be configurable to support different ECUs. I’m considering using one of the few remaining 5V compatible CPLDs as a configurable level shifter.

                                              The funniest part of this project so far was needing to buy a PCI Express parallel port card to interface with my integration test bench: a cheap eBay “Willem EEPROM Programmer.”

                                              1. 1

                                                In the technical realm, trying to figure out how to modularize a 2k LOC behemoth of a D3 charting routine. What does good modularization look like in JS?

                                                In the non-technical realm, realizing that yes, when I eat fatty foods for lunch my productivity drops like a rock, and no, just drinking coffee isn’t enough to overcome that. Since I started eating better, both my mental clarity and my productivity have skyrocketed!

                                                Next step: establish a more rigorous gym routine.

                                                1. 1

                                                  I’ve mostly been taking a break from work stuff during the holidays since Apple’s App Store review team was gone for a week. Some big features are coming out in Pushover after the new year that required a lot of work on the website and both mobile apps, so I’m glad to have a break from it.

                                                  This has given me some free time which I have mostly spent hacking on Endless, a privacy-focused web browser for iOS I made to fill a need I had when switching from Android, where I used Firefox with some plugins. It has reached a state where I can use it as my primary browser on my phone (despite it not having bookmarks yet). I might keep working on it for the rest of this week and finish the remaining TODOs before returning to work stuff next week.

                                                  1. 1

                                                    A lot of the same stuff as last time to be honest.

                                                    I did finish a big chunk of Predictable Revenue, and I’m looking forward to putting some of those strategies into use here at Fogbeam. I did not however, get much reading done in the Introduction to Artificial Intelligence book, so that’s still on my list. And I’m still working on features for our ESN product.

                                                    I also need to finish a migration project, where I’m moving all of our internal infrastructure from Rackspace to Linode.

                                                    My TODO list is actually about 8 miles long, and I have a nearly infinite amount of stuff that needs to be done, so my biggest challenge right now is prioritizing, and avoiding letting things overwhelm me to the point of getting nothing done.

                                                    Do you need feedback, proofreading, collaborators?

                                                    Well, if your firm is interested in improving intra-organization collaboration and knowledge-sharing, and want a unified integration layer that can “connect the dots” between documents, databases, spreadsheets, BPM systems, business events and “open data” then get in touch.

                                                    Likewise if you like Groovy and want to hack on some cool Open Source stuff, drop me a line. We’re always looking for collaborators in that regard. Unfortunately we’re not funded (other than out of my pocket) so I can’t offer anybody a full-time job working on our stuff. But I can say that, for anybody who contributes to the open source project, we’ll do something to look out for you when the day comes that we have money.

                                                    As far as feedback and proof-readers… not today, but I want to rewrite a lot of copy on our website soon, so I may ask for some feedback / critique once some of that stuff is done.

                                                    1. 1

                                                      Ya know, I just realized that what I really need to do is finish this dang Rackspace -> Linode migration. I just got my payment receipt from Rackspace for the month, and was reminded that I’m paying them $167.00 / month, when Linode is about half the price, and for better spec’d VM’s. It’s ridiculous for me to keep paying these guys this much money for no good reason. As of now, I think I’m making that my top priority for the rest of the week.