1. 19

The weekly thread to discuss what you have done recently and are working on this week.

Be descriptive, and don’t hesitate to ask for help!

  1.  

  2. 9

    Current hobby project is a Chrome extension called RecipeFilter. Got tired of reading all the cruft on recipe blogs, so I made this to bring the recipe cards front-and-center. Works on all the worst offender sites I’ve visited so far thanks to some generic conventions many sites observe. Includes a blacklist so you can prevent it from running on specific domains by clicking a button. Next I will add a way of managing the blacklist and get it into the Chrome Web Store.

    I showed it to some guy that seemed interested on HN and of course he has a brilliant plan to monetize it already in place ;)

    1. 7

      I’m going to be attending the NumFOCUS DISC unconference in NYC later this week. I’m hoping to work on establishing a program for interested OSS maintainers in NumFOCUS projects to mentor programmers from underrepresented backgrounds. I believe strongly that a key to sustainability in our OSS projects is onboarding new contributors and maintainers, and I don’t want our projects to be limited by the narrow demographic pool they are currently drawing new contributors from.

      1. 7

        I started using the Getting Things Done system (https://hamberg.no/gtd/) to keep track of the various projects I’d like to work on. So, using the lists that that system encourages me to create, I can report that this week, I am going to spend trying to set up a personal Diaspora instance, to use to proxy messages I send to facebook or any of the other social networks they support. I like the idea of being able to use the externally-controlled social media platforms I am used to, but at a remove, mediated by software I control, and allowing me to ensure that I retain a copy of anything I see or write there. I don’t know how good diaspora will be at this, but that will be something I can judge, once I get a local instance working.

        1. 5

          Floorplan designer/drawing tool for a sims-like game I’m working on. Should be job searching but feeling pretty discouraged about it.

          1. 3

            Same with me. Working on a 15 puzzle solver instead.

          2. 5

            Working on a disease surveillance and alerting system, trying to finish up a new analysis engine and web ui for querying against large varied data sets as well as a proof of concept desktop application in electron (eventually to be refactored into something with a smaller footprint) using a sort of gossip-like syncing mechanism to allow multiple mobile apps and desktop apps to be disconnected from the main servers and still be able to collect and share localised collected data and analysis.

            Other than that, frontline support and loading up legacy data sets for clients doing some proof of concept work in golang.

            Outside of work, studying for Life in the UK test that I have to take in order to apply for permanent residence here in the UK and setting up plan b for if I don’t get approved (will have to fly back to Canada and start paperwork to get wife and kids into Canada…). Working on an iOS kids game for a change of pace and to try a different style of programming.

            So, lots…. probably too much.

            1. 2

              as well as a proof of concept desktop application in electron (eventually to be refactored into something with a smaller footprint)

              This has a tendency to not happen.

              1. 1

                I’m actually going to port it over to QTs WebEngine eventually, i’m writing most of the code in node-gyp with C++ so I can pull the library code over to QT and only using JS for the actual UI rendering. So hopefully that will save me a large chunk of time porting it. I’d build it on QT5 directly, but can’t get the money together for a licence so that I can statically link QT.

                1. 1

                  I’d pay good money for some form of WebRender+Node platform that would run on the big 3 desktop platforms (and stay up-to-date and compilable)

              2. 5

                I’m currently trying to write my own job description, which you’d not expect to be as hard as it is! My role is Head of Architecture and it takes a similar shape to what I think Martin Fowler does at Thoughtworks, but with some different context:

                • at a startup, so the prevailing æsthetic is “stop thinking and start doing”, and it can be challenging to introduce more abstract levels of consideration
                • at a startup, so with a much smaller team
                • at a startup, so with a large amount of nuts and bolts coding as part of my job
                • at a startup, so I don’t actually know what Martin Fowler does, just what he and some other people claim he does

                Also we’ve got a big demo for a prospective client this week on my main project, so I’m uncovering bugs, pushing fixes and uncovering other bugs.

                Outside work: I did a couple of PRs for GNUstep at the weekend, which I’m hoping will not need much cajoling to get landed, and I’m doing the blockchain course at Coursera and have a frustratingly difficult assignment to complete.

                1. 5

                  Work: Implementing this ietf draft on top of this ieft draft on FreeBSD. If I am really lucky I won’t have to deal with any bureaucracy this week.

                  !Work: Probably too much.

                  The GPD Pocket has a whole raft of devices connected to i2c, for some reason I can’t figure out ig4 is unable to use the bus. I need to ask for help on a mailing list, but need to find some time to focus my question. I have a review in to support the gpio controller (which the fan is connected to), if there is progress on that it will become the top task. The back light is controlled by pwm, I have written most of a driver for that, but I can’t get acpi to probe the hid it should be on. That needs more time to look at.

                  Sick of reading datasheets and linux sources trying to track down the ig4 issues I spent this weekend working on bringing up the mt7610u usb wireless driver. I need to figure out why the usb stack doesn’t like my usb descriptors before I can continue.

                  1. 5

                    I’m setting up my base station for amateur radio! I just got a power supply and antenna for my IC-7100, and I’ll be listening on the air soon.

                    1. 1

                      Nice, I always wanted to get into ham radio but never managed to make enough free time.

                      1. 1

                        It’s never too late! http://www.eham.net/newham/

                        1. 1

                          What discouraged me was the sheer amount of real estate a setup would take on my desk. I don’t have the luxury of having a free room to dedicate to the hobby, so my study would have to house all the extra kit, which I simply don’t think it would!

                          1. 1

                            A handheld radio is not much bigger than a cell phone and it’s a good first radio to purchase. No need to buy a huge base station until you’ve gotten more familiar with the hobby!

                    2. 4

                      I am working on a proof of concept for GDPR using a graph database and vuejs. Wednesday I will be speaking about API first CMS at WHO in Copenhagen.

                      1. 2

                        GDPR is going to be a hot topic next year. Is your idea to demonstrate links between data points?

                        1. 3

                          Yes it is! I am preparing a GitHub repo and few blog posts. I will share all when it is ready.

                          1. 1

                            Please do, I’m interested on this matter!

                            1. 1

                              Hello, as promised I have published the first part here: https://blog.grakn.ai/gdpr-threat-or-opportunity-4cdcc8802f22 the second part is here: https://medium.com/@samuelpouyt/grakn-ai-to-manage-gdpr-f10cd36539b9 and I have yet to publish the api example. Code is available here https://github.com/idealley/grakn-gdpr

                        2. 1

                          Are you talking about GDPR at the WHO? Or an actual CMS?

                          1. 3

                            At who I am speaking about Cloud CMS an actual CMS we have implemented where I work, but I am speaking generally about API first CMS’s and the benefits they can bring to a company, especially if you need to publish to different channels.

                            1. 1

                              Have you spoken at any other humanitarian agencies yet or worked at an NGO in a technical capacity before?

                              1. 1

                                I am working at an NGO. And we have implemented it. I agree it requires some technical knowledge, but the benefits are huge!

                                I did not speak at humanitarian agencies on this topic, but I have have in other digital circles.

                                1. 1

                                  Cool, well good luck! I haven’t been to the Copenhagen office before, been to GVA and in-country offices, they only let me out of my cage to see the outside world once in a blue moon.

                                  1. 1

                                    I was also in cage. One day I was invited, my boss said no. I took the days off on my extra hours, and financed myself. Like this trip to Copenhagen. :( But all the rest is fun!

                        3. 4

                          I usually can’t speak about my work, but this week I’m giving a talk to a university robotics club about motion planning. Trying to do more talks in general.

                          1. 1

                            I usually can’t speak about my work

                            Man, that sucks. I hope you’re keeping secrets for the right moral reasons. Too often people do unkind things when they have secrecy.

                            1. 1

                              Could I get a link to your materials? I’ve been writing a blog series on control theory, so I’m curious to see how others teach somewhat similar topics.

                              1. 2

                                Sure. I don’t have them collected yet, but I plan to provide a collection to the students so I’ll send it along.

                            2. 4

                              Yay, I actually finished porting the Oil lexer to re2c, as I said I would last week [1]. I showed some examples of this here [2].

                              This 440 line file: https://github.com/oilshell/oil/blob/master/osh/lex.py

                              turned into 573 lines of re2c expressions. That in turn gets converted into 14,429 (!) lines of C code. However, most of it is switch statements with 256 cases (dispatching on every byte). re2c relies on the compiler to do “switch lowering” [3], which I didn’t know about before a couple months ago (a helpful commenter on Reddit gave some links).

                              GCC and LLVM both do it. In LLVM, they use 3 or more strategies so that a 256-case switch statement isn’t a linear search: finding closed-form expressions, jump tables, binary search, and a hybrid of all the techniques.

                              So the object code ends up at 19 KiB. That’s probably bigger than it should be, but it’s not big compared to the rest of OSH, so I’ll leave it for now.

                              It is faster – 5.2 seconds to parse the largest file (20,000+ lines, 629 KB), down from 8.5 seconds. The unit tests are noticeably faster too!


                              That’s good but the parser is still too slow. Now I am working on the third bottleneck mentioned here [4]. Instead of using reflection at runtime to generate Python classes from the ASDL schema, I’m using textual code generation at build time. This sort of “type-specializes” the code, which makes it faster. It’s also pretty easy, especially since you have the repr() function.

                              I’m pretty sure that will be done this week.

                              [1] https://lobste.rs/s/jq1wne/what_are_you_working_on_this_week#c_um4iqg

                              [2] https://lobste.rs/s/cd5lk4/low_hanging_fruit_programming_language#c_pnmweb

                              [3] http://llvm.org/devmtg/2015-10/slides/Wennborg-SwitchLowering.pdf

                              [4] http://www.oilshell.org/blog/2017/10/24.html

                              1. 3

                                I’ve just started writing a java library for gopher using streams. It really surprised me to see that no such thing existed, at least to my knowledge, not even for basis Java. It’s probably tied to the fact that Gopher as a protocol is quite simple. The goal is to use it in a modern gopher client for android.

                                Sadly I don’t have the time to work on it over the week too much, so I won’t be expecting too much progress too soon - on the other hand, I’ll have time to plan it out during my commute to uni.

                                1. 3

                                  This weekend I worked a few hours to catch up on some laziness on my part during the week. I spent the time optimizing my ci / cd.

                                  Managed to get a upwards of 45 minutes process down to less than 5. It basically involves compiling a haskell app with a docker build image, copying the executable over to a much slimmer docker image and pushing it to aws ecs. No tests yet.

                                  1. 3

                                    I’m slowly chewing through the Haskell Book (about 200 pages in, 1000 left to go.) My goal is to be done with it by Christmas. In the meantime, work has me simplifying some legacy code. Cruising until the Christmas holidays.

                                    1. 1

                                      What’s your review so far? I’m constantly tempted to buy it.

                                    2. 3

                                      I got my implementation of delimited continuations to some sort of closure. Now it’s back to the compiler for Mu targeting a subset of x86. Phew, that digression took up a whole month!

                                      1. 3

                                        The subset of x86 sounds cool. How did you choose the subset? Any reason for 32-bit vs. 64-bit?

                                        Since I just ported the Oil lexer to re2c, I noticed there are 3 nontrivial compilers:

                                        1. Python regex -> re2c expressions (I wrote this in a few dozen lines of Python, using the undocumented sre_parse module)
                                        2. re2c expressions -> portable C code, with switches and gotos emulating a DFA (this is what re2c does. It’s a 20K line C project)
                                        3. The C compiler is necessary to do “switch lowering” on re2c output (see sibling comment). Otherwise, it would be much less efficient than a hand-written lexer.

                                        Each piece is rational to some degree, but the whole thing is bordering on a Rube Goldberg contraption. What I find interesting is that assembly is a better language for DFAs than C ! But it’s not portable.

                                        Russ Cox’s authoritative articles on regular expressions talk about Ken Thompson’s paper, where he directly translated regexes to machine code:

                                        https://swtch.com/~rsc/regexp/

                                        And now that I look again, there is some short and interesting code which I haven’t studied:

                                        https://swtch.com/~rsc/regexp/regexp-bytecode.c.txt

                                        https://swtch.com/~rsc/regexp/regexp-x86.c.txt

                                        These are C programs directly translate regexes to something more like assembly language than C programs. I should try those out and see if they can improve upon my long compiler pipeline. Although I think portability is a strong motivation. I wonder how fast an x86-subset interpreter would be on ARM.

                                        I’m not that familiar with assembly language, but this seems like the perfect project to get more familiar with it. I did it in college, but never professionally.

                                        Anyway I wonder if the output of these programs could run on your subset? Might be a fun test case.

                                        Other examples here:

                                        https://lobste.rs/s/cd5lk4/low_hanging_fruit_programming_language#c_pnmweb

                                        1. 3

                                          Fascinating links, thanks! Let me take a look and get back to you.

                                          How did you choose the subset?

                                          https://github.com/akkartik/mu/tree/master/subx#readme

                                          Any reason for 32-bit vs. 64-bit?

                                          a) It’s still easier to find tutorials online for x86 rather than x86_64 :) I started out knowing absolutely zero about this ISA. (My grad school work was with the Alpha 21264, which was a really clean design but is now sadly obsolete.)

                                          b) As I said in my link above, inside x86 is a really nice, clean 32-bit ISA struggling to come out. The memory model is flat, no segments. Segments made a comeback in x86_64. 64-bit versions of instructions are often bolted on using a global mode to decide what word length to use. Ugly!

                                          c) I still try to create 32-bit VPSs in the cloud these days, because they’re a little more memory- and therefore cost-efficient. No acres of bits in wasted in sign-extending addresses. I haven’t needed more than 2GB yet in a single program. (And good thing too, because such VPSs get expensive fast!)

                                          d) It’s not clear to me that there’s an exclusively 64-bit ISA inside x86_64. There definitely is an exclusively 32-bit ISA inside x86 that doesn’t rely on 8-bit or 16-bit instructions. That means that if I focus on 32-bit instructions now, that doesn’t preclude the possibility of supporting 64-bit instructions in future. No work should need redoing. (Except instruction decode for that global mode.) Whereas supporting 64-bit would well involve implementing at least some 32-bit instructions anyway.

                                          1. 2

                                            OK interesting… yeah this is something I have to learn more about. I bought this book recently, and it does cover x64:

                                            https://www.amazon.com/gp/product/9332573905/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1

                                            Actually they define something like “Y64”. But I just got it and haven’t gone through it. One annoying thing is that it uses AT&T syntax and everybody says to use Intel syntax.

                                            But otherwise it seems to be a good book.

                                            But yeah there seems to be a lot more material on 32-bit x86 out there.


                                            BTW I compiled and ran both of the examples above now… they don’t run! One gives me a segfault; the other seems to hang. I only looked at it for about 5 minutes though.

                                            1. 1

                                              Code samples not working is indeed one of the perennial issues here :/

                                              As is the two syntaxes. I had to gain some fluency with both, because the manuals use Intel syntax, and anytime I want to experiment with as I have to use AT&T. But in my code I stubbornly refuse both options and describe instructions in words. Like the comment on this line.

                                            2. 1

                                              Correction: My reason b) is incorrect. Thanks https://mastodon.social/@vertigo/99097883285637446

                                            3. 2

                                              Anyway I wonder if the output of these programs could run on your subset? Might be a fun test case.

                                              No, not as such. Here’s the instructions used in https://swtch.com/%7Ersc/regexp/regexp-x86.c.txt

                                              static
                                              unsigned char header[] = {
                                                  0xC8, 0x94, 0x10, 0x00,                         /*      enter   $400, $0                */
                                                  0x8B, 0x55, 0x08,                               /*      movl    8(%ebp), %edx           */
                                                  0xB8, 0xFF, 0x00, 0x00, 0x00,                   /*      movl    $0xff, %eax             */
                                                  0x31, 0xC9,                                     /*      xorl    %ecx, %ecx              */
                                                  0xE8, 0x00, 0x00, 0x00, 0x00,                   /*      call    _next                   */
                                                                                                  /*_next:                                */
                                                  0x83, 0x2C, 0x24, 0x05,                         /*      sub     $5, (%esp)              */
                                                  0xA8, 0xFF,                                     /*      test    %al                     */
                                                  0x75, 0x02,                                     /*      jnz     _L1                     */
                                                  0xC9,                                           /*      leave                           */
                                                  0xC3,                                           /*      ret                             */
                                                                                                  /*_L1:                                  */
                                                  0xE3, 0x0A,                                     /*      jecxz   _L2                     */
                                                  0x49,                                           /*      decl    %ecx                    */
                                                  0xFF, 0xB4, 0x8D, 0x70, 0xFE, 0xFF, 0xFF,       /*      pushl   -400(%ebp,%ecx,4)       */
                                                  0xEB, 0xF4,                                     /*      jmp     _L1                     */
                                                                                                  /*_L2:                                  */
                                                  0x8A, 0x02,                                     /*      movb    (%edx), %al             */
                                                  0x42,                                           /*      incl    %edx                    */
                                                  0xE8, 0x0A, 0x00, 0x00, 0x00,                   /*      call    _code                   */
                                                                                                  /*_fail:                                */
                                                  0xC3,                                           /*      ret                             */
                                                                                                  /*_nnode:                               */
                                                  0x8F, 0x84, 0x8D, 0x70, 0xFE, 0xFF, 0xFF,       /*      popl    -400(%ebp,%ecx,4)       */
                                                  0x41,                                           /*      incl    %ecx                    */
                                                  0xC3,                                           /*      ret                             */
                                              };
                                              
                                              static
                                              unsigned char footer[] = {
                                                  0x4A,                                           /*      decl    %edx                    */
                                                  0x89, 0xD0,                                     /*      mov     %edx, %eax              */
                                                  0xC9,                                           /*      leave                           */
                                                  0xC3,                                           /*      ret                             */
                                              };
                                              
                                              enum    {
                                                  CMP     = 0x3C,
                                                  JNZ     = 0x75,
                                                  CALL    = 0xE8,
                                                  JMP     = 0xEB
                                              };
                                              

                                              Extracting just opcodes and deduping, we get (with deep links into the Mu codebase):

                                              • 0x31 (xorl)
                                              • 0x83 (sub 8-bit immediate; 32-bit version is 0x29)
                                              • 0x3c (cmp 8-bit immediate with %al; 32-bit version is 0x39)
                                              • 0x41, 0x42, 0x49, 0x4a (incl/decl; use add or sub instead)
                                              • 0x89 (movl to register)
                                              • 0x8a (movb 8-bit)
                                              • 0x8b (movl to memory)
                                              • 0xb8 (movl 32-bit immediate to %ecx)
                                              • 0xa8 (test; like and but doesn’t write result, only condition flags)
                                              • 0x75 (jnz to 8-bit offset)
                                              • 0xeb (jmp to 8-bit offset)
                                              • 0xe3 (jecxz; jump to 8-bit offset if %ecx = 0; Mu will rely on previous instruction to set the flag instead)
                                              • 0xff (pushl among others)
                                              • 0x8f (popl)
                                              • 0xe8 (call)
                                              • 0xc3 (ret)
                                              • 0xc8 (enter; CISC instruction now considered inferior to combining RISC pushes)
                                              • 0xc9 (leave; CISC instruction now considered inferior to combining RISC pops)

                                              The missing instructions aren’t difficult to add, and they’d often improve instruction density. But it’d make the compiler a little more complex to worry about instruction density. So they’re out for now.

                                          2. 2

                                            Increasing my understanding of Jenkins’ server-side while I hamfistedly attempt to adapt/re-implement some classic functionality in a way that will play nicely with Blue Ocean.

                                            1. 1

                                              I am working in different open source projects with other members of my small company:

                                              • We are writting an erlang’s database client wrapper that reconnects to the database after a netsplit/database restart/etc.
                                              • We are working on a simple drop in WebRTC server written in erlang. Audio/Video calls are really easy to implement client side using WebRTC. Howeve it is not that easy as it seems to implement server side. You need to implement a STUN/TURN server apart from the signaling server. We are trying to package everything so that you clone it, run it or even download a docker image and you can implement your own WebRTC server for your iOS/Android app or webpage.
                                              • We started working on a small Erlang dependency that automatically joins erlang nodes to the same cluster if they are in the same subnet.
                                              1. 1

                                                $CLIENT: trying to prioritise a heap of urgent fixes in their old codebase. Errors calling partner API’s, SQL Injections up the wazoo, you name it.

                                                $INTERNAL: Putting together a minimalist phpDoc to Markdown API generator, after discovering the official phpDoc project is a fucking nightmare to try to fix bugs on.

                                                1. 1

                                                  At work, I have passed my probationary period, and as a reward was gifted with another person to manage (or “coach”, in Flipp parlance). We have a great relationship already, so this should be pretty painless, but it is time consuming, and my likelihood to be landing patches in our various repositories is diminishing. Oh well – I took this gig because I had lost the joy in writing software, so it’s a bit rich to complain if they take me at my word?

                                                  Outside of work, we threw an American Thanksgiving party, and 50 people showed up. We’re still cleaning the house. #1 Daughter (2 ½ yrs) has decided that she no longer wants to sleep in baby jail, so now her mattress is on the floor, “like a big girl.”

                                                  In-between work, I’m putting together a small group here to go through The Haskell Book, and I’m trying to get more people to do Advent of Code 2017 with me (assuming it shows up).

                                                  1. 1

                                                    Due to upcoming deadlines at University I’ve sadly been unable to do much. My small Kernel Project gained a paging system and virtual memory, my blog engine got an overhaul, I’m now aiming for something integrating into Gitea. Otherwise, I’m just way to busy to focus on anything else, sadly.

                                                    1. 1

                                                      I made my first, very small, Chrome extension which simply adds a repo’s creation date as a badge on Github, next to Star, Watch etc. It’s only really designed for myself but I figured maybe someone else could get some use out of it too.

                                                      I gotta get a thumbnail and squish a few minor bugs first but other than that, it’s pretty much good to go I think.

                                                      1. 1

                                                        I’ve been working on a new website for the Code & Supply Scholarship Fund ahead of #givingtuesday and a holiday fundraising event we’re planning for the end of the year. CSSF is a non-profit that funds tech conference experiences for underemployed & unemployed software folks and members of underrepresented groups in the Pittsburgh software community. The site might go live before people awaken tomorrow if my team reviews my code before they hit the hay tonight!

                                                        While I’m pretty adamant about manually entering transactions into my plaintext accounting system, there are some parts of it I want to automate because of the high probability of error in manual entry. One such area is converting Fidelity’s CSV exports from its 401k management tool to ledger format. I’ve been working on it here and there. It’s starting to get to the point of usability. The goal is for it to output a ledger-formatted transaction log given the CSV and require only a little manual tweaking. https://github.com/colindean/ledger-fidelity-401k-csv-parser