1. 18

This is the weekly thread to discuss what you have done recently and are working on this week.

Please be descriptive and don’t hesitate to champion your accomplishments or ask for help, advice or other guidance.

  1.  

  2. 13

    Creating this unholy abomination. That is, Dwarf Fortress streaming over X11 forwarded through websockets in a browser. Yes, it’s horrifying, and works better than it has any right to…

    It’s built with xpra, which I’ve gradually been fixing bugs in. Multiple people can spectate (or even control) the windows at once, which makes it great for continuation games (i.e. where multiple people do rotations on controlling a fortress).

    Input and display latency are shockingly small. Dwarf Fortress is especially well suited to xpra’s lossless compression modes, because most of the screen is the same color.

    Currently working on embedding it in a NixOS container so people can’t find anything too useful by poking around the filesystem with the Qt file browser embedded in Dwarf Therapist. From a security standpoint, I’m fully expecting that arbitrary code execution is possible in the container and am just trying to contain the damage if it does occur. This is a game people have built computers in, apparently

    Also, I’m trying to figure out a good name for the project. “Armok Web Services” maybe? ;)

    [edit] here’s a video of it working: https://streamable.com/uax2f - most of it is just me showing different parts of Dwarf Fortress (not very interesting) so skip around. Point is that it’s working with pretty high fidelity and low latency over the internet.

    1. 1

      Update: Wow, Nix containers are easy to get working! I’d probably still be nervous if random people from the internet had access to it, but now the limited group that’s testing it can’t read my /etc/passwd with Dwarf Therapist.

      The project is officially called Armok Web Services, and has a home now: https://github.com/numinit/armokweb

    2. 8

      After feeling some pain about how complicated all software, IT and Enterprise architecture toolsets are, I got this idea of a dead simple architecture sketching tool, that’s based on just nodes and named lines. The input for this tool is a text file that contains triples in the form of subject-verb-object, like so:

      # comments via shebang
      # declare nouns (for better typo recovery)
      internet, web front, app server, DB, Redis
      # declare verbs (perhaps not necessary and may be dropped)
      flows, proxies, reads/writes
      
      # subject-verb-object are separated by more than 1 whitespace (" " or "\t") 
      # somewhat like Robot Framework does it
      # prepositions (to, from, at, etc.) seemed redundant, so not using them
      internet     flows          web front
      web front    proxies        app server
      app server   reads/writes   DB
      app server   reads/writes   Redis
      

      I’m not married to the syntax yet, but it seems fine after a few iterations of experimentation. A tool could then read this and produce a nice graph via graphwiz. And you don’t really need a tool, you could do these sketches on a piece of paper in no time. But when you want to store and share the design, a text file + tool is a good combo.

      Tried to write a parser for this language in Rust’s pest, but that was a bit of a pain – pest’s error modes are quite odd. Seems like when my parser has a problem, Pest blows up on position 0 every time. Perhaps I’ll just do a dumb line-by-line parser by hand instead. And I’d like to implement in Crystal rather than Rust. Also thought about doing it in Swift, but that doesn’t seem to work so well outside of MacOS/Linux (I’m thinking BSDs here) yet so nope.

      Best part? The tool’s name is Architect Sketch.

      1. 5

        Reminds me of mermaid. It has shortcomings but has worked for me most of the times I’ve had to diagram something over the past few years.

        1. 4

          I’ve been working on something similar myself, with the goal of actually deploying infrastructure on it. I think this is a good avenue to explore, and pairs really well with rule engines and datalog systems.

          1. 2

            This is something I’ve been thinking about a lot as well.

            For my use case, I just want to dump, document, or understand how my multitude of web apps, daemons, and IoT devices all interact with one another.

            I had built a rough prototype with ruby and shell scripts using similar syntax, but I couldn’t ever get Graphviz to generate a “pretty” layout for the whole thing.

          2. 7

            I’ve discovered that pretty much any project will take me 3x as long as I think, so I’m taking a more modest approach to try and complete one thing a week as entirely as I can. I’m tired of sitting on projects for years and telling myself, “I’ll work on that later” for eternity.

            Anyway, last week I completed my documentation/tutorial for emulating a z/OS mainframe, https://famicoman.com/2018/06/28/emulating-a-z-os-mainframe-with-hercules/

            This week I hope to take a stab at documenting my PBX setup.

            1. 7

              I bought a new laptop, a 6th gen. Thinkpad Carbon X1, and I’m setting it up with Void Linux. (Replacing a Macbook Pro – I’ve been wanting to switch to a non-Apple laptop for a while.) My “laptop” Ansible playbook drove most of the setup, but I still need to make it suspend when I close the lid.

              A couple years ago, I wrote some blog posts about using Ansible to configure laptops (1, 2). I’ve been meaning to update my example repo – while the concepts are all the same, Ansible has had some interface changes over the years. It’s usually good about messages saying e.g. “sudo: was deprecated, use become:”, but it’d be nice having an up-to-date example, and it would document getting Void running on this Thinkpad.

              I’m going to make Void packages for a couple things I’ve installed locally – mscgen, Alloy, TLA+, and drivers for my printer/scanner (a Brother HL-L2380DW).

              Aside from that, I’ve been working on a C library for parsing/scraping ambiguous data using Earley parsing and parse forest grammars. (Related, I gave a talk at Papers We Love SF about the Earley parsing paper a couple weeks ago.) This has been off and on for a while, I’m still adjusting the API to account for important use cases it supports beyond general parsing. I’m also working on an LL(1) parser generator, a clone of djb’s redo, and learning radare2 by working through some reverse engineering exercises.

              1. 6

                I just finished up .NET bindings for my technical analysis library. It’s something I’d been meaning to do for a while.

                1. 6

                  For $WORK, we have just deployed a new version of our entire technology platform! Our clients are doing their first major migration starting on the 2nd (today, for most folks reading this), adding 1,000 users per week throughout the entire summer. This is migrating from a physical colo’d C# monolith stack to a hybrid Rails/C# microservice architecture spread across AWS and Azure (with some of it calling back to the older stack). It’s the culmination of many months of efforts by every team in the company.

                  I’m nervous, but I’ve been dedicated to load testing it using Selenium, Firefox, and AWS instances – 1,300 of them to be precise. It’s handled that simultaneous load well enough. I hope to finish collecting all of the details and publish a blog post on how I set up this pseudo-bot-net of mine.

                  My personal infrastructure at home and other personal projects have been put on hold since I lived and breathed this preparation work for the past couple months. This week I hope that between refreshing every monitoring page in the network I can spend some time kicking the development back into gear.

                  1. 6

                    I’ve been working on my solo entrepreneur SaaS product, NewBusinessMonitor. I’ve been adding cosmetic touches and I’m trying to make the site work better on a mobile device, as to my surprise, most of my traffic comes from mobile devices.

                    I’m adding a feature where users can save searches they’ve made, and recall them later. After that, I want to add historical financial data to each company.

                    Tech: Haskell, Elm, Nix.

                    1. 5
                      • For $CLIENT1 it’s mostly following on from last week

                        • The initial model migration was pretty straightforward.
                        • This week I’ll hopefully clear up any remaining questions/uncertainty about the new model system for the other dev’s on the team, and proceed with migrating some more of the ‘simpler’ models across to Bamboo.
                        • Plus of course, regular bug fixes/prod incident handling
                      • For $HOME we’ll probably need to make some decisions about internal light placement soon, so the builders can run wiring inside the panels for us. We’re into wet season now, so it’s nice to know it’ll be all sealed up soon.

                      1. 5

                        Finished subscriptions handling in my app, rolling out TestFlight beta version for now. So this week is cleaning up the UI, code, adding more elaborate error handling, cosmetic touches. Hope to make my first submission to App Store by the weekend.

                        Work, studying and experimenting with Precision Time Protocol and its possible use in a product.

                        1. 5

                          I’ll try to train any regression model on my cycling performance data (HR, cadence, power, speed) to see if I can accurately predict my power data without a powermeter.

                          Early tests have shown my powermeter from my smart trainer is too inaccurate (~10 %) and my indoor training sessions are not a good source for my training data, I’ll see if real word data from a friend will improve results.

                          Has anyone done anything resembling this ?

                          1. 5
                            • prepping campus course feedback tool for 7 summer sections.
                            • trying to add Redis as a cache store to the above project
                            • Trying to launch APP, https://beta.presidency.ucsb.edu later this month?!
                            • Move APP Solr instance to SaaS service.
                            • hang in there …
                            1. 5

                              I am working on https://gridgenerator.com

                              I am making this tool because I am not a designer and struggle with design tools. I do enjoy simple shapes and patterns. I hope that this can be useful in any way to others. I also added the possibility to review your creation process, like this: https://gridgenerator.com/p/13

                              This week and the next I am going to try to add a few islamic patterns grids, inspired by Allambra. Also I need to improve the usability and onboarding which are inexistant (coder ux ftw).

                              1. 5

                                Improving the docs and releasing molten 0.2.0. The main focus of this release is going to be Python 3.7 and OpenAPI support (the framework now bundles the Swagger UI and can automatically generate OpenAPI documents).

                                1. 3

                                  Molten looks neat. Especially request validation based on schema classes looks like it would eliminate so much validation code. Nice work!

                                  1. 2

                                    Thanks!

                                2. 4

                                  Work:

                                  • lots of people on holiday, so probably only doing bare essentials to keep things running
                                  • still some HPe machine issues, at least they are sending a technician now

                                  Home:

                                  • further improvements to my infrastructure automation, I got some things done last week so that’s nice.
                                  • I started some menial task automation on my phone via macrodroid, I’ll continue this week
                                  1. 4

                                    Working on Merit’s new Proof-of-Growth system. I’m in the testing phase. I just got back from vacation and am ready to tackle testing and releasing this. The stages of testing are first on the regression chain, then testnet chain, and finally general release with a delayed feature switch.

                                    1. 3

                                      I’m trying to write a “sudoers evaluator” using Linux namespaces. Something like:

                                      • Create child with clone() and flags CLONE_NEWUSER, CLONE_NEWUTS, CLONE_NEWNS
                                      • CLONE_NEWNS so I can mount the input sudoers file over /etc/sudoers inside the namespace
                                      • CLONE_NEWUSER so I can remount sudoers.so to fiddle with the permission bits to let the namespace execute it (currently stuck on this: sudo: /usr/libexec/sudo/sudoers.so must be owned by uid 0)
                                      • CLONE_NEWUTS so I can set the hostname of the namespace so that you can then evaluate sudo -v -h <hostname>

                                      The goal is if I have a sudoers file like:

                                      Host_Alias FOO = bar[1-52-3]
                                      user FOO = (baz) quux
                                      

                                      You can run with ./sudoers-eval ./fake-sudoers-file --host bar23 to ensure that host bar23 is not picked up by the expansion of bar[1-52-3].

                                      Hope I get something working by the end of the week. I started the project in C, switched to Go (where I was using the reexec package: https://github.com/moby/moby/tree/master/pkg/reexec) until I decided that if my goal is to understand the primitives behind Docker, importing Docker helper libs would hinder my education, so I switched back to C.

                                      1. 3

                                        I’m writing the first server to test the new file protocol of Jehanne.

                                        Also looking for a strong copyleft license for the new protocol that can express/preserve it’s nature as a gift.

                                        1. 2

                                          At home: inspired by the Arch and Nix discussion a few days ago, I decided to try Nix on macOS and NixOS on my workstation with root on encrypted ZFS.

                                          Liking it so far and nixifying all the things. I use a very thin system configuration now and do most of the heavy lifting using home-manager.

                                          I am not sure whether I’ll stay yet. I like the Slackware-like simplicity of Arch. On the other hand, working with Nix feels like you have superpowers, especially with nix-shell and direnv’s Nix support.

                                          1. 2

                                            Just returning to $DAYJOB from vacation, and really not enjoying it. Mostly trying to do administrative tasks to push things around a bit. It’s also very boring in the office because I’m in Sweden and everyone’s on holiday.

                                            Fortunately, the side hustle is really interesting and is keeping me entertained. I’m doing some work on quality measurement tools for C#, and it’s pretty enjoyable.

                                            Finally, I had this idea to build a Cocoa-native Slack client using the RTM API, because I hate Electron so and there are a few communities on Slack that I really enjoy participating in. I am trying to figure out if I made such an application, if I would be able to sell it at all to other like-minded mac users to help offset the time investment I was putting in to it. (And of course I’d aim for *nix and Windows support via GNUstep.)

                                            My current worries on this are a) I’d run out of steam after having invested some time in to it, b) Slack would screw me by disabling the required API support, or c) Marzipan would enable the Slack iPad client to be easily ported and eat my lunch.