Feel free to tell what you plan on doing this weekend and even ask for help or feedback.
Please keep in mind it’s more than OK to do nothing at all too!
Continuing work on my remote iterator server!
Progress is really coming along, and I’m really excited about it.
There’s a major overhaul that’s in place, so probably by late tomorrow the protocol and basic commands should work as expected. Then I’ve gotta document everything because while there’s tons to do from a technical standpoint, in about a week it’ll be in a place where I’ll really need feedback from people to know that the ux and concept is growing in a useable way.
Can you give an example of a type of iterator that you have envisioned for remits?
Sure! The basic concept is you push messages to a persisted log, and then can iterate over it.
Say you have a log that contains events of different types that happen in a system, but you have a service that should only respond to a certain kind. You could define a filter iterator over it and have that service consume from it, so it would only get those events it cares about.
Say you want only certain information about that certain event type. You could create a map iterator ontop of the previous iterator we talked about, to transform the response for each element. Similarly you could use a reduce iterator to get aggregates or stats and always pull the “latest” value of the aggregate from the stream, approximating a streaming query.
Iterators will be just-in-time by default, but I want to allow an Indexed Iterator so that you can persist the actual results to disk if you are going to query multiple times and don’t want to redo the work.
Right now the only iterator types are Map, Filter, and Reduce. The callback function passed to each is defined by the user in Lua. The combination of those should be sufficient to express most queries people would want to do. I really like the simplicity of that. The project’s still super young though so I can see an alternate universe where we have zips, filter_maps, flattens, window functions, and stuff like that…but I’d really like to see people’s use cases and experiences before growing the api under a hand-wavy guise of expressiveness.
Did you have a precise use case in mind when you started building this or was it more of a shower thought?
Originally, it was for a few small projects that required something like Kafka but didn’t need the full setup and complexity. Managing Zookeeper and multiple JVMs for a small project wasn’t a good use of time. I wanted something easier to maintain and setup for non-Netflix-sized use cases.
Now I work at a large Fortune 500. We have insane bloat and complexity stemming from not being able to easily “query” a stream. We have hundreds of Kafka topics, or on another team 60 Kinesis streams for one project. That separation is mainly because different services need different information and they still need to transform it on their end. I hope to reduce the complexity by centralizing where you store, query, and transform the data. By being able to optionally realize the iterators to disk, I hope to also minimize the amount of computation done during a fan-out style setup by turning it into a simple “bytes in/bytes out” operation.
Sorry for the wall of text. But I hope it answers your question! If you have any more questions, or comments I’d love to hear them!
No that wall of text was exactly what I was looking for. I’ve a very small project I’m working on (monopoly game) where I originally wanted to have the source of truth be the event stream, and have a queryable caching layer on top of that that represents game state.
Gonna try Portal for the first time
Learning to unicycle.
Woohoo! gonna be a lot of fun and a lot of swearing xD Did you buy one or borrow from somebody? :) Good luck and enjoy! :)
My dad got it as a leaving gift from a past job, and since then it’s pretty much just sat collecting dust. I was meant to have A-levels this year, but the current situation has meant that essentially I’ve just got six months of nothing, so figured I’d go through all the stuff we have and learn to do it. Hopefully it’ll be a lot of fun :)
Tomorrow morning: volunteer at the local food pantry. We changed the rules to reduce contact: the staff fill carts with a predefined set of items, clients pull up in their cars and we unload the carts into their cars. It’s a bummer because the clients don’t get to choose, but it’s the best we can think of under the circumstances.
I’ve decided to write a Spotify client using the Spotify package I’ve been working on.
I originally wrote the package without having a need for it (other than it’d be neat to control Spotify from Lisp), but I recently switched to FreeBSD, and the only option for using Spotify there seems to be spotify-tui, which is a little clunky and doesn’t support some features I really want, like finding similar artists.
I’m still brain storming what it’s going to look like. All of the functionality will be usable from the REPL in Emacs, but it also needs to be usable by StumpWM so that I can have media keys tied directly to playback controls, and I’d also like some kind of UI for finding similar artists, editing playlists, searching, and things like that. Maybe a simple LTK dialog with mini-controls, and a web UI with Hunchentoot as the backend for finding new artists?
Lots to think about…
This sounds cool.
My goal is to spend less time doomscrolling
Upgrading my Home network setup to new shiny Ubiquiti Components.
What are you migrating from?
A almost burned out Asus N66R Wireless AP+Switch, a Netgear Switch, and TP-Link Router.
I really needed this upgrade. I am migrating to two AP AC LR, an EdgeRouter PoE 5, and a EdgeMax Switch ES-16-150W
There’s a few good options. Maybe playing with microcontrollers. Maybe working on writing a personal/small-scale wiki in Rust. Maybe working on Rust gamedev library stuff.
I’m thinking of working through some ECS stuff in Rust to get my head wrapped around that pattern as it has been embraced in the Rust game dev community. I’m going to try to model some simple games, like maybe some casino games just to flex data modeling muscles.
Rust is in a pretty decent place these days for 2D games, there’s lots of good libraries and frameworks. My ECS of choice right now is hecs, since it gets right to the meat of the issue without a lot of the framework fluff that legion and specs have. I’m not sure ECS is necessarily the best structure for something like casino games though. Where it really shines is games where you have a lot of independent things following similar but not exactly the same rules; think something like Space Invaders, or a tower defense game. It’s also not a one-size-fits-all solution for every purpose, I see a lot of new game programmers struggle trying to shoehorn collision detection and graphics into an ECS, when those are generally big enough special cases to warrant a departure from the model.
Parenting, mostly. My kids who can read have received an introduction to Scratch through some game. Now I’ll install Scratch on their computer and they can start coding stuff i guess.
I looked at Crystal a bit this week, i got disappointed when i couldn’t run it on my machine without Docker, so I’m thinking of heading in a different direction. The learning made me realize i missed Ruby, so i might head to Elixir because it has a similar “vibe” and the tech behind it is solid. Might also give a try to installing Jitsi Meet on my local boxes, but at this point it feels like a heavy heavy gun for my occasional needs. I might just resort to a Prosody image instead. Maybe i should just formulate clearly what i would want to replace from the Big Corporations Of Doom first. I’m not sure i like Signal.
I am finally not overloading my watch with interrupts from the accelerometer so maybe some more work on that to be able to tap and interact, I realize now I love application notes. I also have an assignment coming up so that requires more attention.
Besides social distancing… I am going to try to read glTF file in Rust using the JSON parser and 3D render on screen.
Ah yes, also I have to finish this illustration, which is a sort of continuation of the other illustration in
What program are you using for drawing?
When on Windows: Photoshop. When on Linux: MyPaint + Krita + Aseprite. This illustration in particular on Photoshop.
Finishing off Half-Life Alyx.
I was also looking at creating a small web app to interact with the API powering my gym’s food tracking app. It was pretty trivial to pull apart, and some of the mobile UI isn’t greatly intuitive so I might as well make my own
What are your thoughts on the game? No spoilers please! Which VR set did you use? There seem to be a few VR “lounges” in my city, but none of them seem to have Alyx (yet?… hopefully…) :( maybe I’ll have to try checking again later this year…
I used an HTC Vive which I actually bought in preparation for Boneworks. Funnily enough, I still haven’t played Boneworks and was never expecting a VR Half-Life title
I just finished it before and it’s pretty amazing. Not to use a cliche term but there are definitely some aspects of the game you can only experience in VR since they wouldn’t translate well to a regular keyboard and mouse playthrough. Control wise, you could sort something out but you couldn’t achieve anywhere near the same level of immersion you get with VR. Being able to (physically) pick up items, reach out, duck behind cover while firing above it all adds up.
I can pretty easily say I wasn’t disappointed and it definitely serves as the template on what an all-in VR title can accomplish
I’ve been working on a simple redshift/f.lux replacement for Linux based on Ted Unangst’s sct. You can give it your location’s latitude and longitude and leave it running and it’ll transition between suitable temps: https://github.com/amir/sctd/
Thinking to install new leadscrew on my lathe. It needs a different size support bearing: will see if I can make it on the lathe itself. Also, a small silver brazing job to fix a cracked steel part.
Playing with Shapr3D on iPad to make small tweaks to a neat project me&my mech eng colleague made in free time. Won’t say what it is as to not jinx it, should have first test prints by Wednesday.
And as any spring, plenty things to clean up in the garden.
Sheltering in place, and configuring some web serving stuff which I hope will eventually turn into a self-hosted Jitsi instance. Zoom is nice and all, but it has some privacy issues.
Sewing face masks for my family. Also working on my static gallery generator. Hopefully going on a hike tomorrow.
Sewing face masks for my family. Also working on my static gallery generator. Hopefully going on a hike tomorrow.
Please, consider that manual sewed masks provide a fake sense of security. They do not work to stop virus spreading since microholes in most textiles are too big. Sorry for the article in Spanish, but it’s the source where I read it
Thanks, I appreciate your concern, but there’s a bunch of research showing that masks are one of the most effective interventions to prevent spreading infections. They are not perfect nor sufficient and need to be handled with care, but the same is true for washing hands and nobody advises against that. We are “playing” a numbers game and decent masks improve our chances as long as we are aware that this is all they do. I will be very surprised if guidance on them doesn’t change soon (as it already has in many places).
My personal experience with them is that it is very difficult for me to forget I have one and hence more difficult to forget not to touch my face. I’d wear them even if that was their only effect.
Enjoying the empty outdoors with excercising, NVMe drive migration, hopefully making a printer rack, and a bunch of other little things.
Changing the theme of my current blog
A quick video to explain some of the stuff I’ve been playing around with the past week and a rewrite of a chapter/essay on Test-Driven Infrastructure
Continuing to avoid people.
I’m organizing a video conference games night - trying to make the best of the lockdown in the UK / social distancing that we’re having to deal with right now.
If I get around to it, I might try to figure out an alternative to the (very efficient but not cross-platform) nvALT. I’ve tried a few options including sncli and deft-mode, but haven’t found anything that’s quite like the real thing. Oh, yeah, and I’d love to find an option with great (fully keyboard driven) link support. Perhaps I’ll read up on Zettelkasten, which appears to be “in” at the moment.
Also rebased my FreeBSD armv8crypto AES-XTS patch and it looks good now. Might add SHA-2 next.
I just started to port hyper/h11 (I/O free HTTP/1.1 implemetation) from python to Zig.
The objective is to slowly but steadily learn more about Zig, HTTP/1.1 and network programming in general.
If I produce anything, it will be available in this repository.
Have a nice weekend and don’t forget to excercise especially if you are stuck at home 🌴
Continuing work on stork. I’ve rewritten the system that weights each search result; hopefully this weekend I’ll be able to look at each word in a search query and weight results based on how many words in the query a given result matches. I’m looking a lot at Rules for Autocomplete (2019) to have a sense of what I’m building towards.
Continuing to quarantine. Work got busy all of a sudden. Watching Little Fires Everywhere on Hulu and remembering how much I liked the book.
The stork link seems to not work for me…?
Yup, I messed up the url of my own project. I meant to write
(Although while I’m here, I should mention that I finished the new highlighting and sorting features I was hoping to get done!)
Backing up stuff on my on both my laptops. I’m planning to wipe and do clean installs on them (ArchLinux from 2013) and (NixOs from 2018). I expect the backup and new setup to take most of my weekend. Also plan on reviewing some Haskell code I’ve left dormant for too long
I’m working in a desktop project using Java and Swing, and am looking for a good guide on how to implement the MVP pattern into this application, so I can handle everything more or less than I do in Android. Having and architecture is really good for the escalability and testability of the application, plus you keep everything better organized!
PS.: if someone can provide a good guide on how to do what I want to acomplish, please let me know by replying to this comment!
Creating a memorial site for my deceased cats. This was prompted by the death of my almost two year old cat Max by a careless driver. Also staying in isolation as much as possible & washing my hands.
I’m looking for job, and as part of the recruitment process I got a homework from one of the potential employers, so this is gonna make me occupied for a few days…
I’d like to change my $workplace Prometheus pollers to Thanos for some additional feature. It’s very rare that I think about $work over the weekend but these days I find it helps keep my mind off the pandemic.
How or why does one change the poller? I thought Prometheus only exposed the interval.
I just mean installing a separate set of Prometheus (+ Thanos) pollers. That way all the scrape targets will get double-scraped (one by the old poller, one by the new poller), and after a retention period goes by, I can shut down the old cluster.
The reason why is there are too many scrape targets (I use consul_sd exclusively). Our Consul catalogues have just grown too big for single Prometheus pollers. Using Thanos, I hope to set up N Prometheus pollers to shard the Consul catalog, and Thanos is that missing link which allows queries to be executed against all N Prometheus shards.
How good is the “with” keyword!
Honestly? Periodically checking https://wordometers.info/coronvirus/ and various news sites for updates. I can’t help it. For anybody getting anxiety over those figures I’d have to recommend looking into mindfulness techniques, as well as maybe some texts on stoicism; the enchiridion, the stoicism text is actually quite accessible, with translations available on gutenberg.org.
Other than that, I hope to get some writing done as well as baking some breads. I’m slowly getting into bullet journalling, and I hope that it helps me get organized as I start this new job of mine. I’m also taking the extra time I have, now that I’m working from home, to start studying economics proper.