1. 3

    $work:

    • I’m on research week, so more symbolic execution and future of smart contract stuff
    • some report editing
    • fixing up some client tooling
    • looking into some F# stuff I’m seeing in the space

    !$work:

    • I removed 369 lines of type parsing code from my compiler, which resulted from a simple grammar change I made
    • need to finish some more work on the match form
    • I’ve started stubbing out a new CTF I’m working on, a historical CTF with historical machines & languages
    1. 2

      Any more info on the historical CTF? That sounds really interesting.

      1. 2

        so I’ve written a historical CTF once before: Gopher, a modified RSH, and MUSH running atop Inferno, which was pretty interesting.

        For this one, I’d like to have a MULTICS/PR1MOS-like system and a VMS/TWENEX-like system that players must attack and defend. The code would be written in languages appropriate for those two systems (like a DCL-clone, some Algol clones, and so one), with flags planted throughout. It’s a lot of work, but I think the result would be really fun, if quite challenging for participants (new languages, structures, protocols).

    1. 6

      I’m working on a PL project in Rust at work, which will be fun. This week I’ll be adding some niceties such as strings to the VM.

      1. 2

        Oh? What type of PL/VM?

        1. 1

          One of the things I’m not thrilled about with this project is that I can’t say much. :/

      1. 3

        I don’t know what he’s talking about with this:

        I haven’t waded into known strengths of Rust like the FFI and the concurrency model!

        Trying to do things concurrently in Rust made me stop trying to use Rust. Having to completely redo the i/o and suddenly having my selection of libraries Balkanized made continuing in Rust for a hobby project not worth it.

        1. 9

          I imagine Bryan Cantrill is in the class of developers who evaluates languages/frameworks/libraries in terms of their readability and maintainability. Probably because a large portion of his career has been doing maintenance programming.

          When measured along those axes balkanized libraries and the occasional refactor hardly matter. Especially when Rust makes those refactoring quite a bit safer than other languages with it’s type system and borrow checker.

          1. 1

            I heard thread safety is one of Rust’s selling points. What kind of concurrency issues did you run into?

            1. 2

              Presumably he’s referring to the hoops you have to jump through because of Rust’s thread safety guarantees.

              You can’t just use shared mutable state like you otherwise would have, and until recently there were multiple competing (and mutually incompatible) approaches in use.

              1. 2

                It has the same problem as C/C++ where I actually have no idea what kind of threading model any given library is going to use. Spawn a thread per request? Thread pool? Some actor framework? Callbacks? If it’s using a framework, which one? Do all my libraries play nice with it?

                I think futures are in the stdlib, but async/await isn’t in stable. The rust book touches on futures, but mostly focuses on implementing your own ThreadPool, and not anything about how to organize futures-using code.

                1. 3

                  Yep futures in std, and async/await are currently in nightly and will be available in the near term.

            1. 1

              Finishing up a meta data scraper for YT videos in an archival team up with archive team. Putting some finishing touches on my init system, and getting set up at my new job.

              1. 2

                Working on finishing a single binary init system in Rust. Other then that I’m exploring youtube APIs so I can make some modifications to a distributed scraper I wrote. Planning to help an archiver try and scrape all meta-data off youtube.

                1. 5

                  Finishing up an infra move at work, putting the final touches on an HTTP server I wrote to be my personal homepages backend, and starting work on implementing my own ethereum virtual machine im rust.

                  1. -8

                    Zero sympathy for those pwned.

                    1. 4

                      Why? Admittedly they should check and change the default config. But having no auth by default is a terrible design decision just to keep backward compatibility.

                      1. -3

                        Why? Because using etcd in the first place is probably a dumb decision, and if you (for some bizarre reason) actually need it, then anyone who calls themselves a “sys admin” should be prudent enough to not install software that exposes all of your most private and important credentials to the Internet. And if you do install such software, you should most definitely be aware of it and configure it properly — before it takes down everything.

                        If you’re not competent enough to do that, you have no business being a system administrator, you are a security threat to anyone who hires you.

                    1. 15
                      • Spent a lot of time polishing some of my libraries to get them to 1.0 (with help from others).
                      • Rewrote my CSV library in Rust. Pretty sure it’s one of the fastest general purpose CSV parsers. It’s at 1.0-beta. Close to a 1.0 release.
                      • I managed to publish 1 blog post (on aforementioned CSV library), which keeps my pace going.
                      • Helped with the SIMD stabilization effort for Rust, but others have helped a ton too. We basically crowd-sourced writing implementations for all the various Intel vendor intrinsics (there are ~thousands) and it worked really well!
                      • Wrote a small CLI data entry tool to import data into a ledger format. I still need to copy & paste tables from PDF files though on occasion. /sigh
                      • Spent a significant amount of time working on a low level information retrieval (think Lucene, but one layer down) library. I have nothing to show for it so far. It’s hard.
                      • Began a giant refactor of Rust’s regex library internals that will probably take years. This year I almost finished a rewrite of the parser. The goal of the refactor is to make it easier to add optimizations and break apart the internals. A secondary goal is to support stream matching. I don’t know if I’ll succeed in either endeavor!
                      • Ported several hierarchical agglomerative clustering algorithms from C++/Python to Rust for fun/work. I’m proud to say that I matched performance and used zero unsafe.
                      • I started work on a principled approach (read: not some random mish mash of Python scripts) to parsing the Unicode Character Database, with the intent of using it to (mostly) provide complete support for level 1 Unicode support. I’m also casually building a tool to browse/search the UCD. I had been using a Go tool that someone else wrote, but it’s… surprisingly slow.
                      • Continued active maintenance of ripgrep. Interestingly, I didn’t add any new major features this year (perhaps UTF-16 support qualifies, but in terms of code changes, it was very small). I do still have some planned that I had hoped to get to this year, but that ain’t happening. :)
                      • I gave my first talk on ripgrep at the Boston Rust meetup. It went well.
                      • I discovered Dan Carlin and listened to all of his Hardcore History podcasts during my commute.

                      At $work, I did lots of stuff too. A big part of this year was spent adding access control to our system. I’m also working on a collaborative graph editing system (think Google Docs but for knowledge graphs) which has a ton of interesting problems in it.

                      1. 1

                        Oooh, I’d love to hear more about this data retrieval library. What formats do you intend to support? Been looking for something more lightweight then lucenne.

                        1. 2

                          Hmmm. I don’t know what you mean by that. It has its own format, just like Lucene has its own format. It isn’t going to support Lucene’s format is that’s what you mean.

                          The idea it’s to provide a library for creating and querying a single IR index. In Lucene terms, this is at the level of a single segment. It uses fsts for the term index, uses skip lists for the postings and all that. It is mostly just a prototype at this point. I’m struggling with what the API should look like. For example, you invariably need to talk about multiple segments for merging, even though I was trying to avoid that (merging needs knowledge of the format on disk to be fast). This in turn means making assumptions about identifiers. So it is a bit of a knot at the moment.

                          1. 1

                            What I meant was something more lightweight resource wise then running a jvm.

                      1. 2

                        Currently working on writing a quick character generator for Cryptomancer in rust followed by releasing a more genericized crate for anyone who wants to do quick character generation for their trpg of choice. Otherwise it’s just setting up monitoring and backup scripts at work.

                        1. 3

                          Continuing on with my automated penetration testing kit. Golang has been a god send for easily parallelizing connections and threads. Working on adding the fun stuff which will including worm like capability to spread among a network after gaining a foothold.

                          1. 2

                            I use dokuwiki myself for long term storage, and google keep while mobile. Have a script set up to query google keep and add the content to my dokuwiki instance every night.

                            1. 2

                              This seems like a nice solution… mind sharing the script somewhere?

                            1. 5

                              I tend towards archiving, sorting and catalouging all my data. So I have full site rips of several science fiction blogs as well as textfiles.com, Bruce Scheiner’s blog, and quite a few others. Along with:

                              UC Berkely’s online Video lectures All of C3 All of Blackhat All of Defcon Several TB of full tv series Several thousand movies Twenty thousand plus books Full archives of byte, mondo, 2600 magazine, etc. And much, much more.

                              1. 2

                                Admirable dedication.

                                I have to add that I hate when there are online video courses I bought that I cannot easily download, like from Thinkific. I know I can watch it anytime I want, but what if the site will go down, or the creator closes his account with all the courses she or he was selling? I don’t like it (just like I don’t like subscription model in software, because if I buy something, I want to have it accessible perpetually and obviously offline too).

                                I know it’s to prevent piracy, but typical thing with anti-piracy protections is that they make lives of users harder, while pirates will somehow grab the content anyway if they’ll be really willing to do it.

                              1. 2

                                Just mundane work stuff, finishing up containerizing a few coin daemon’s for usage with our mining pool software and setting up rancher-nfs to use as a remote volume store for various client versions of their coins individual blockchain. Other then that I’m hashing out what instructions to include in my VMs instruction set, trying to find the nice cross section between minimal and featureful.

                                1. 3

                                  Plasma Mobile (and Hildon, because I’m an N900 nostalgist) is IMHO essential from turning this from just a toy that engineers play with to something that could affect change by allowing you do daily drive these old phones. (Unfortunately, I have concerns with that - mobile hardware wasn’t on the same plateau desktop computing is.)

                                  1. 3

                                    I’m one of the two maintainers for the N900 on pmOS, and you can bet that I’ll be working on polishing Hildon (and hopefully Plasma Mobile) on this sucker once we nail down some of the more fundamental functionality (e.g. telephony)!

                                    1. 1

                                      I’ll seriously consider flashing the N900 and giving it another go, Unfortunately, the N900 is somewhat crippled by only having 256 MB RAM, (despite having 32 GB of flash - in 2009!) and I think there’s not a whole lot of ecosystem going on for Hildon nowadays. (Plasma Mobile is small, but will likely grow.)

                                      1. 2

                                        Don’t quote me on this, but I think I have heard in the pmOS chat that Plasma Mobile will work on devices with 256MB of ram. One of the original Plasma Mobile demos was done on the N900 a few years back.

                                        On a side note, I ran XFCE on mine with pmOS and another user had Sway going on Wayland. Mainly “just because we can”, though none of us are using those environments at the moment since most of our time is dedicated to further development of pmOS.

                                        1. 1

                                          Hildon is just X11 though, so any of the qt/qml/gtk stuff being built for other environments can work there :)

                                          Yeah, the RAM limits on the N900 are the only reason it’s not still my daily device, since web pages have mostly grown to require at least 1GB RAM to load :P

                                        2. 1

                                          I can live without telephony so long as the data modem works ;)

                                          1. 1

                                            Totally understand. Anything to do with the modem on this thing is still under active debug/development!

                                            1. 2

                                              For sure. Keep up the good work!

                                        3. 1

                                          I’m very excited for a Hildon revival! I got my current device (Blackberry Q10) in part because the UI felt enough like Hildon to be familiar.

                                          1. 2

                                            I have a Q5, but I’m unimpressed with BB10 as an OS. The UI feels only like a shallow clone of Maemo - it lacks the nuance that made it click to me. The Android runtime is also flaky and the ecosystem dead.

                                            Silver linings though: It has a great Exchange client!

                                            1. 1

                                              Yeah, I’m desperate to replace my Q10 with a Pyra!

                                              1. 1

                                                I wanted a Pandora when they were new, but I kinda lost my enthusiasm for GNU/Linux along the way. (I still kinda want one anyways, though.) The thing with the Pyra is that it’s more like a laptop or even the pre-N900 NITs - a WiFi based tiny “MID” or gaming thing. Cellular might be possible on Pyra now, but the form factor seems clumsy for a phone replacement.

                                                1. 1

                                                  They can still be had on ebay, bought mine from the US retailer about… 5 hears back. Probably gonna grab the Pyra when it drops, if only as a nice backup terminal and media device.

                                                  1. 1

                                                    Well, I suppose it depends what you use a “phone” for. If you primarily make voice calls, it’s probably not for you (though with a bluetooth headset maybe even for that form factor matters less?)

                                                    For me, I’m mostly texting (well, XMPPing, really), some surfing, some ssh’ing

                                          1. 1

                                            Ah, one of my favorite days of the year. :)

                                            1. 3

                                              Been working on learning more of Golang after my refresher on C. Been working on a minimal iftop clone, and have also started preliminary work on implementing a ssh client. Planning to start porting my minimal C compiler clone to Go as welll after my ssh client is done. So much to do, so many rfc’s to read.

                                              1. 8

                                                Happy New Years you crusty crustaceans. May 2017 be ever in your favor. :)

                                                1. 7

                                                  Aye, it’s a shame it was shut down. Haven’t seen a nexus for discovering and enjoying new and obscure artists like it for years. Although I’m sad to see it go, I’m excited to see what site shall take it’s place. Especially secure in the fact that a large amount of the original content released on the site is saved and planned to be reuploaded elsewhere.

                                                  1. 7

                                                    in particular, I hope that what rises from these ashes is truly distributed and decentralised!

                                                    1. 5

                                                      Soulseek has about the same amount of music as What.cd (and more) but there’s no quality control. You may be able to find album X but not album X in FLAC.