1. 19

This is the weekly thread to discuss what you have done recently and are working on this week.

Please be descriptive and don’t hesitate to champion your accomplishments or ask for help, advice or other guidance.


  2. 13

    Last week, I made my first non-trivial program work!

    libra is a huge achievement for me, in many aspects. It’s contributing to a project I love, it’s learning how to do a bunch of stuff, it’s feeling accomplished, all in one.

    This week, I’d like to work on its interface. It’s still not very useable by other people, and considering it’s made to be used by other people, that’s still a bug!

    1. 7

      I’m starting the course https://www.coursera.org/learn/financial-markets-global as per the recommendation comment here: https://lobste.rs/s/2rwddp/what_is_on_your_2018_reading_list#c_3houma

      And continuing my massive art photography project as noted last week.

      (of course, the above is contingent on how much free time I have outside of work).

      1. 4

        I’m starting the course

        Same here! Let’s see if I make it to the end of this Coursera course. That would be… the first one?

        1. 3

          Want to team up and each make sure the other stays on track?

      2. 7

        I wrote the user-space emulation of non blocking IO for Jehanne, and now I’m looking for more tests.

        Please share links to POSIX C code that uses non blocking IO.
        Any small program that exit(0) iff non-blocking IO works will help.

        1. 2

          The screenshot link on your front page is down.

          If I have time some time this week, I’ll swap out 9front for Jehanne for my occasional “Plan 9” hacking explorations. I’ve been curious how things are going over there.

          1. 2

            It’s not down, it’s sadly “work in progress”… that is: I had no time to write the page down.
            Sorry… I can either code or edit the web site, and I usually prefer to code. :-(
            I’ll try to give it a shot asap.

            Thanks for trying Jehanne!
            Just beware that it is really unstable, by design.
            And I’m going to change it deeply in 2018. So do not use in production. :-)

            OTOH, 9front is the most stable evolution of Plan 9.
            I imported a lot of their improvements, but if you are looking for a Plan 9 system, you might be disappointed.

            I mean: if Plan 9 from Bell Labs were Homo Sapiens, 9front would be Homo Sapiens Sapiens.
            Jehanne would be a race of Elves.

            1. 2

              Just beware that it is really unstable, by design.

              I’m very aware, but 9front is already in my “for fun” bucket, with things like Haiku and DragonflyBSD, so that’s fine. (Honestly, to a point, it’s almost better, because it means the community is probably actively having some fun and not entrenched in process yet, and the last thing I’m looking to do in my spare time is to learn or contribute to Yet Another Unix that’s almost but not exactly like what I do for work each day.)

            2. 1

              Just added a screen shot at http://jehanne.io/pages/screenshot.html

              Also, the link in the front page should work now.
              This is marketing, dude! :-D

              (it has been work in progress for more than two years!)

          2. 6

            Making lots of progress toward stabilizing my rust lock-free bw-tree! Hopefully I’ll have an alpha out soon :) The goal for the next week is to have stable disk utilization while under extreme contention and fragmentation. Now that crash testing and interleaving tests have teased out much of the low-hanging fruit, it will soon be time to turn my attention to dynamic instrumentation for teasing out exotic race conditions :] If anyone is curious about lock-free high performance stateful systems in rust, feel free to ping me, and I’d love to spend some time teaching potential collaborators. There are a ton of really juicy database features that I’d love to delegate to other people who are curious about learning how to build them!

            1. 6

              I’m actively working on multi-core support for theft again. While I’m still integrating the code for shrinking failures, running individual tests parallelizes nicely: in some early/casual benchmarks, running 10,000 trials of a CPU-bound test on a 4-core system indeed runs ~4x as fast as before. Shrinking should also be well-suited to parallel exploration. Once that’s working, I’ll do more substantial benchmarking, on a system with 40 cores.

              It may be a couple more weeks before the next release, but in the time I’ve found for it, I’m making steady progress.

              Incidentally, taking an existing project and restructuring it so that individual units of work can be relocated to different processes, moved around a network, etc. is an interesting exercise. A lot of implicit decisions become explicit. Thinking about what actually needs to be passed along in messages to the next phases (rather than everything staying in-scope by default, due to sequential computational inertia) draws attention to implementation details hiding in plain sight. Also, this restructuring is also a bit tedious in C, because, well, C isn’t Erlang. Pattern matching and tagged tuples would help a lot…

              I started learning x86-64 assembly a couple weeks ago, and started doing Project Euler for practice. I’m still only a couple problems in, because I’ve focused on better understanding details like the costs of different approaches. The early problems have just the right level of difficulty for useful self-study. It also gave me some context that was useful for Meltdown & Spectre-related investigations last week.

              Finally, I am working on a few submissions for the IOCCC. :D

              1. 5

                I got enough motivation on my big-deal side project to get it started, got it started, then hit an issue: I’m working in a HURD* VM and don’t have a browser set up in it, so I can’t add my ssh key to GitHub to publish my code. The host OS is Windows so I don’t really understand how I might file-share my key between the two. Fixing it is just a question of apt-get, but I decided at the point that I hit that problem it was a good place to park it, and will be dealing with that this week.

                Work: we’ve got two distinct products that solve two similar problems, and the goal in the short term is to turn those into a single product where we can sell one or both features. I’ve designed the capability configuration and explored its impact on the architecture, it’s time to put that into practice. Additionally one of the teams I work with has failed to meet its delivery commitments for the last few sprints, and I’m investigating why with the people, putting changes in place, and communicating those changes around. And I seem to be involved in a few job interviews we’re hiring!

                *why? I want to use its model of message-sending for IPC. My thing is building an Erlang-style message passing model, but without doing all the threading in user space on a VM that is only supported by a couple of languages. If you can build your message-send function in C, then it can be used by anyone whose language includes a C FFI which is nearly everybody. I think, but do not know yet, that I can do the same on Protocol Buffers, and will investigate that once I’ve used Mach (which I understand better, i.e. at all) to express a solution to the problem.

                1. 3

                  I fixed the paragraph 1 issue by installing lynx in my HURD VM and adding the SSH key in the github UI.

                2. 5

                  Had an idea at the weekend, instead of building base images for my standalone SmartOS hypervisors by hand I could lean on packer via the Joyent public cloud. Does the same thing, and costs a few cents per image build. Given I’ll build one of these per quarter most likely, I can live with the cost 😆

                  Once that’s done, then I just need to move everything on the home server across to a new supported base image and make sure puppet is managing what it needs to on there. Sounds an awful lot like work to me 🤔

                  1. 4

                    For the next couple of weeks I’m going to continue to hack on the Thymio, Aseba, and Jetson stuff I have for my Masters.

                    I’m working with one other on an (academic) year long project to implement SLAM to map unfamiliar environments, and then use evolutionary algorithms to identify the best strategy to achieve various goals. Since the bot hardware is cheap and cheerful, the main challenges we’re facing is developing any kind of useful SLAM output, and navigation through it. For simulating and developing improved strategies, the plan is to handoff to the Jetson strapped on top and use the GPGPU cores (OpenCL) to be quicker than the PIC device in the bot itself.

                    1. 1

                      Cool projects. I don’t know what SLAM is in this context. Turning it into a link would be helpful.

                      1. 3

                        They probably are talking about Simultaneous Localization and Mapping

                        1. 1

                          Oh, OK. That makes sense. Thanks.

                    2. 4

                      Building an AI to play Ambition. I tried gradient descent / backpropagation but the results were mediocre (barely better than random play) and, given that most of the 608 inputs aren’t likely to be relevant– I doubt that holding the 7 of diamonds matters in most cases– I’m going to try NEAT to evolve a sparser (and faster to train / use) network that, I hope, will perform better.

                      Farisa’s Crossing: had a bit of a writing slowdown due to family emergency over the holiday, plus work pressures. Getting back into it. Going to read over previous draft in order to plan the last third of the book (although I basically know how the story ends) in the revised version. It turns out that writing a literary-grade novel takes 5-10+ revisions.

                      1. 4

                        Most importantly, data recovery. Windows Startup Repair was of the opinion that my LUKS drives need to be formatted as GPT and nuked all backups of the LUKS header along with that. So I’m recovering the most important stuff from backup atm (I did loose an ancient backup of mostly lower sentimental value and a small data collection which is a bit annoying, I only backed up /home really…). I’m still torn on whether to nuke the disks completely and setup a LUKS+RAID1 or try to recover anything.

                        Once I’ve got everything recovered, I’ll have to deal with a few bugs in the configuration language I’m writing, a few new features have been suggested to me and I’ll implement them this month for the v0.9.5 milestone

                        I’m also looking to rewriting some parts of my toy kernel to be more robust, especially the global kernel state (without the stdlib, having a globally mutable static variable is slightly annoying to do in Rust, even moreso if you don’t have heap when it needs to be setup).

                        I also think about starting a curator/archive project, I’ve got some design/architecture ideas for that, nothing concrete yet. I might experiment a bit in what to write it in too so I’m flipping through various webframeworks and languages.

                        1. 1

                          I also think about starting a curator/archive project

                          I’m interested in retro computing and have volunteered with / hope to volunteer again with some computing museums in the UK. Anything I can help with regarding requirements/UX?

                          1. 1

                            Probably not yet, it’s still in the early stages, as mentioned, I haven’t even decided on a language or framework yet.

                            It’s not really targeted for retro computing, more in the direction of archiving and curating user content from websites, it could help me to compensate my data loss from above (it’s all online but I curated what I archived)

                        2. 3

                          One of the things on my bucket list for 2018 is creating a small project and let it have at least 1$ in revenue. Not profit, just revenue. I’m creating a list of some ideas but they all seem pretty bland and hard to monetize. Currently I’m pondering to do something with a tool to help schools to automate some things. This is the first time I’m doing something like this and I must admit it’s harder than I imagined.

                          At $work it’s going to be a though week again, lots of things to do and unexpected issues will turn up again. I’ve been working there for almost 2years now and most weeks are still full of challenges.

                          For reading, I’m planning to start with ‘1984’.

                          1. 4

                            I would not recommend starting with an idea, however good it seems. I tried it, it doesn’t work. It’s much better to start with the right kind of research to find out what people will pay money for. This site has a lot of useful information: https://stackingthebricks.com

                            1. 0

                              Sites like that recommend a course of action that really smells a lot like bait-and-switch to me. It might not be technically a crime depending on your jurisdiction / industry, but it doesn’t seem particularly honest.

                              1. 1

                                Whatever do you mean? Have you read much of this particular site?

                                1. 1

                                  That one is actually the least like what I described in that particular niche, so I guess I sounded kinda dickish. Most those “start a business” information products are just instructions on pretending you have a bunch of products, and then somehow pulling the one that gets the most bites out of your ass in a month. I find the idea of advertising something I don’t have and probably will never build extremely distasteful.

                                  1. 2

                                    To be clear, I think it’s nothing like what you are describing, and it’s unfortunate that you chose to make the first comment, possibly putting @kamme and others off having a look, and (by association) casting me in a negative light with talk of crime and dishonesty. Please take more care next time!

                            2. 4

                              I’m trying to build some small profitable services too, I’ve been finding it tough indeed (but I broke $1).

                              1. 2

                                IIRC, the advice that startup factories like Y Combinator give on this issue is to identify something in a market segment that’s already painful or just could be better. You pick something you understand pretty well first. Then, you figure out how it could be done better where you or other people would enjoy. Then, you might try a startup idea based on that. They or whoever it was that I read said this leads to success the most often. Double true if it stays within domains you have a lot of experience in so you’ll avoid gotchas.

                                The schools is a good example of it. The people who already are in schools or deal with their IT functions have a good idea of what will or won’t fly in that market with buyers, teachers, etc. They’re the best people to do a business in that area. They’re also the best people to listen to if you want to get in it.

                                1. 2

                                  That’s why I’m thinking about schools, I know quite a few teachers of various age groups and different social and economical levels. Every time we have a discussion about their work I will find a couple of things that could be improved.

                                  The hard part here would be sales. Schools are not known to be flexible and teachers often have no saying in purchases. This means I would have to target principals and administration, and this is not going to be easy to sell as they often have very tight budgets.

                                  I’m leaning towards doing 4 or 5 smaller projects this year and probably target different audiences for each. I know this will not be Y Combinator approved, it’s an excersise to create a bit of a habit and learn different skill sets.

                                  1. 2

                                    This means I would have to target principals and administration

                                    Good point. Start reading up on their decision-making in acquisition. Talk to them a bit as a person that knows there’s issues with current products looking to find ways to improve things for both them and teaches. You get it straight from the kind of people you’ll be selling it to. You can also note what you’ve learned from teachers’ problems for effectiveness. I’m sure you were already going to do that, though.

                                    If budgets are tight, maybe SaaS model with an upfront discount on licensing where you tell them you’re just covering setup and support costs that year. This isn’t my area of expertise, though. I just know they would want minimum transition cost and/or offset. Watch out for lock-in, too, of anything that’s in an old system.

                                    “I’m leaning towards doing 4 or 5 smaller projects this year and probably target different audiences for each. I know this will not be Y Combinator approved, it’s an excersise to create a bit of a habit and learn different skill sets.”

                                    Oh yeah, I’m not saying go YC or anything. I was just saying experts warn to stay in stuff you have real-world experience with. Many looking for any money-making project make mistake of straying too far from that. You’re not. It’s good you’re aiming for a few projects for self-improvement with possible financial gain a bonus. Good luck on that for sure. :)

                              2. 3

                                Working on an acoustic impact detector for industrial environments. Analyzes multiple mics at a time, and can configure particular audio signatures to violate on, based on various audio features.

                                1. 2

                                  At work, I’m aiming to try my first patches to some of our Elixir components, which will be interesting.

                                  At home, I’ve been working on a series of blog posts about “the everywhere computer,” which is basically the idea of having your entire computer environment available anywhere, but not by just relying on web apps or needing to SSH into a specific machine. My goal for the week is to get a miniseries on file systems in this space (currently IPFS, Minio, KeybaseFS, Perkeep, and Upspin) done enough that I can launch it with some confidence I can maintain a once-a-week post cycle, but we’ll see.

                                  1. 1

                                    Please post a link to those posts to one of these threads when they’re up. It’s an interesting area, and I can’t be the only other person who’d like to know more about it.

                                    1. 1

                                      You might find the Nirvana Phone interesting on the desktop-anywhere angle. For a whackier take, there’s the person that switched to Windows 3.1 on all devices to do photoshop on the cheap.

                                    2. 2

                                      Currently working on writing a quick character generator for Cryptomancer in rust followed by releasing a more genericized crate for anyone who wants to do quick character generation for their trpg of choice. Otherwise it’s just setting up monitoring and backup scripts at work.

                                      1. 2

                                        Finding time to write down notes about things I’ve learned so I don’t forget them over the years. Inspired partly by https://lobste.rs/s/lubstu/personal_wikis_lobsters_tale, but also trying to make myself a little more disciplined about it by making the notes public.

                                        Finished the first one earlier this week, about methods to use traceroute etc. for determining where data goes across the globe, and the many difficulties involved.

                                        Then I plan to use some time on my 11hr LHR->SFO flight tomorrow to read some Lovecraft (At the Mountains of Madness in particular) which somehow I missed when I was younger.

                                        1. 2

                                          At work, I’m focusing on getting more tickets handeled, and having an interesting time exploring our CRM tables

                                          Outside of work, I’m learning Godot. I’ve wanted to do game development as a hobby, and have done so for game jams in the past, but Godot seems to be the right mesh of a reasonable GUI editor that isn’t as heavy as unity, and a scripting language that’s fairly decent. Up until now I’d been using Love2D a for game Jams, so Godot’s 2D toolkit has been an interesting change of pace.

                                          1. 2

                                            Work: Checking out Meson to see if I can stop writing project files for all the systems I have to maintain, as I try to create a library to share with 3rd parties. I remain unconvinced, but perhaps. Any other multi-platform build file creation programs you recommend? Google’s gyp and GN just seem like typical Google overkill.

                                            Home: Still hope to check out pony, he says for like the 5th week in a row…

                                            1. 2

                                              Made a new release and a brand new readme for https://github.com/dapphub/seth — our command-line tool for Ethereum.

                                              I also started playing around with “planning” — using old school STRIPS-style “good old fashioned AI” planners, as well as Ceptre & Celf to do planning in the form of linear logic solving — as part of some hobby research into user interfaces based on automatic plans.

                                              Finally I’m continuing thinking about smart contract language design.

                                              1. 2

                                                Two approaches I found interesting when studying classic AI were Procedural Reasoning System and Firby’s Reactive, Action Packages. You might enjoy making some knockoff of them.

                                                1. 1

                                                  Thanks, I’ll look into those!

                                                  It’s interesting how hard it is to even locate any working software implementing the paradigms of good old fashioned planning…

                                                  1. 1

                                                    I don’t know if that well-understood phenomenon is interesting or depressing given I was one with high hopes for it all. The job was just way harder than people thought. Our brains are really just amazing. :)

                                                    Well, that’s why it all got defunct. The availability part might have come from how so much was done in commercial/defunct LISP’s, Prolog’s, Poplog’s, and other obscure stuff on odd machines or OS’s. Then, just older stuff being hard to get a hold of in general. They used weirder tooling than most, though.

                                                    1. 1

                                                      I’ve understood that there is an annual competition for automated planning.


                                                      All the software seems pretty obscure, though!

                                                      There are several numbered major versions of the PDDL action language, but it’s nearly impossible to find implementations of any of them…

                                                      With linear logic programming, it seems like quite a well-developed research topic, but the only implementations I’ve found are Celf and Ceptre, neither of which are really polished (and Celf’s last commit was years ago)…

                                                      Actually I’ve read some on the deep critique of symbolic intelligence, especially in Hubert Dreyfus’s Being-in-the-world, which is philosophically fascinating and seems accurate. But automated logical reasoning, even though it can’t “come alive”, still seems like a primary reason to even have computers.

                                                      At a summer school of GF (Grammatical Framework) last year, I got the impression that formal grammar models is still very much an active research topic. Hybridizing it with neural models seems like the frontier. I assume there’s some similar hybrid program in automated logic. Grammar, semantics, and logic are so intertwined anyway…

                                                      1. 2

                                                        It’s really interesting you said the competition because that was in me comment preediting to cut size. I originally wrote another trend in AI was that narrowly-focused AI tech that got really great stopped being called AI any more. The examples I had were computer vision and planning noting that the new sub-fields had their own conferences. They get obscure, too, when that happens.

                                                        I recently saw the planners in conferences last year looking up where the field was at for a scheduling use-case. We have so many that perform well. Scheduling, esp timetabling, is such a big problem across the US you’d think the algorithms or FOSS solutions would get more attention.

                                                        So, the obscurity on AI side makes sense to me given AI Winter and Narrow AI Is Not AI Effect. The part that’s strange for me is operations side not utilizing them effectively.

                                                        One more thing: the machine/deep learning side has a momentum going so strong it’s redefined AI to mean that in minds of a lot of people. Quite a few AI majors I met at ML-focuses places didnt think traditional AI was even being developed any more. They never saw it. Silo effect from drowning out.

                                              2. 2

                                                I’ve been working on a motion controller for a giant vertical pen plotter (with a work area that tops out at around 400m^2), and I had been trying to get everything working with a BeagleBoard Black, but last week I tried to bring up the board only to discover that BBB are a pain in the butt and there are undocumented restrictions on how certain pins on the board can be used.

                                                After all of the frustrations from last week, this week I’ve decided to scrap the BBB entirely and try to do a design with the ESP32, which I’ve been meaning to try out for a bit anyway. I finished the schematic yesterday, parts should be here by the end of the week and I can hopefully bring up the board and get some steppers stepping over the weekend!

                                                1. 2

                                                  At work I’m back-porting some changes to a patch release.

                                                  Outside of work I created a Fast Fourier Transform visualization using a grid of Julia set fractals. The code is on GitHub and there’s a sample video.