1. 28

This is the weekly thread to discuss what you have done recently and are working on this week.

Be descriptive, and don’t hesitate to ask for help!

  1.  

  2. 11

    Implementing coroutines for C. Most existing implementations use setjmp()/longjmp() or ucontext.h to implement coroutines, but I decided to forgo those and roll my own in x86 (32 bit) assembly (for Linux). It’s simple (only 32 x86 instructions) but it wasn’t simple to write. I’m planning on doing a write up and putting the code up on github later today. It was a rather fun project in learning how to swap the stack out from underneath the program.

    1. 3

      Do you just wanted to learn or do you have some other motivation for implementing this? I once read a comment about various C coroutine libraries, that the context switching part that is often implemented in assembly could be shared between many of those libraries.

      What do you plan to implement? Do you plan to implement thread pool for syscalls and file i/o? Or M:N mapping?

      1. 3

        Just wanted to do it. I use Lua coroutines for work (handling network requests) and I thought it might be fun to get something similar working in C. I’m not currently planning on using this for anything serious (especially at work where we use Solaris under SPARC, and I have zero experience with assembly under SPARC). It was just a learning experience.

        1. 2

          Lua, SPARC, and network requests? color me very intrigued!

          1. 3

            Processing SIP messages to implement caller name ID (returns an actual name associated with a phone number). Since most of it is parsing, I used LPeg to parse the SIP message and form a request to the backend that does the actual work (including the name lookup, which uses DNS (NAPTR records)). It runs on SPARC because of regulatory issues dealing with power (NEBS compliance, since we are, technically, part of the phone network).

      2. 2

        And finally published: C coroutines

        1. 1

          Whoa, neat.

        2. 1
        3. 9

          As expected my child has been born. It is a lot of work, but so far a bit less then I expected. Subject to change :)

          And as it is with plans I had only the time to think about what I was planning to do.

          I still want to rehash the event loop in my text editor.

          I was thinking about the zero-copy pipes for a image manipulation pipeline environment. Now I am sure that I have to do this differently, because in my preliminary thoughts I forgot about a simple case of an image size change. Maybe I could connect filter processes with Unix sockets and pass file descriptors to the shared memory. Or maybe it would be better to think about this more like modern GPUs do the pipeline?

          1. 1

            Glad everything went well! Any advice for someone whose first is about to arrive?

            1. 5

              As always: there is no silver bullet :) You will receive lot’s of advice that was applicable to one’s child, but it is not applicable to yours. Generally: take it easy. Parental comfort is very important, because parents must remain sane. It’s easy when everyone is healthy and you have financial comfort. I have no much experience otherwise.

              We have a very good midwife and one specific piece of advice that was very helpful was about feeding. It is best to burp the baby few times in one feeding session. When the child starts to fall a sleep during breastfeeding you should try to keep it awake. But eventually burp the baby and if it wants more give it more. After 2-4 times (around 5 minutes max) the baby will be satisfied. Because of interruptions the baby will try to feed quicker next time.

              Good lesson here is to have a very good midwife :) Ours is open minded. Her advice reminds me of Observe, Orient, Decide, and Act decision-making technique. An advice that does not include a feedback loop is most probably not a correct one. An advice saying that you should always do this 5 minutes straight or do that 3 times a day is most probably wrong. For example our baby is quite big (4.3 kg) and the midwife said that we should consider it as an one month old.

              When my wife will recover more we want to try Baby-led potty training.

          2. 7

            It’s been 3 years since I launched http://krgspl.com and it’s been humming along, with about 2 or 3 games played per day. It needs a revisit and serious upgrade. I have had some features in mind for awhile and want to make it mobile friendly. First things first - this week I will migrate from the old RethinkDB to Postgres…which is quite a large effort, then move to a newer version of node and ubuntu. I will add TLS behind an nginx reverse proxy, which I have done before but never with websockets, so that should be fun. After all that refactoring is done I will be able to add features in the coming weeks.

            1. 2

              I did not play, but a few screenshots or example replays would be much appreciated! Usually when I see registration I pass indefinitely… Especially when I don’t know what I’m registering for.

              However the idea of imperfect information is very exciting one. So maybe I will try it later. I had few basic ideas for real-time games where you could see opponent moves with latency (not like high ping). Or where your enemy could cut your supply line and you would know it with significant delay if you were not prepared. Or where light is travelling slower the further it is from you.

              1. 2

                Thanks for the feedback! Indeed, the first change after the upgrade is to remove the signup requirement…you’re definitely not the first to ask. Replays are publicly available now but hidden past the signup, so passersby cant find them - another thing on the list to fix.

                When the next version is ready I’ll post it here and perhaps we’ll play a game. One thing I do know is that it is addictive and has a cultish fan base. If people weren’t playing I wouldn’t be making the upgrade!

            2. 7

              I’ll be teaching Programming Languages at CSU, San Bernardino next quarter, and so I am working now to put together all the materials for the class. I’ll be using “Programming Languages: Application and Interpretation” by Sriram Krishnamurthi as the textbook, and the lab will include Haskell, Rust, Java, and Prolog. I’ve got most of it planned out already. What’s left is to put together the reading schedule, flesh out the complete labs, and finish the course project description.

              1. 5

                I’m giving a talk on P4 [0] at the Berlin SDN/NFV Meetup in two weeks, so working on getting that presentation perfect (yes, it is the enemy of done, but if there is one place my perfectionism shines it’s presentations. also I need to start cultivating EU-based clients, customers, and work)

                I ditched my home-grown plans and decided to fire up PNDA. [1] Only this Palo Alto firewall that’s intended to do NAT for the cluster of servers I’m assigned to use is just….not cooperating. I’ve spent 8 hours debugging, reading, and trying every stab in the dark. It’s been years since I’ve had to open a TAC case…and so now I eagerly await their call in a few hours to finally set me straight.

                I’m hoping to get some “blog” posts going on extending netmiko (a python library for dealing with network equipment CLIs in a more programmatic fashion) via a simple, but in-use in production example. It’s also time to start cleaning up and extending the TextFSM templates I have for Brocade NetIron devices. It’s a glaring gap in NAPALM [2] device support, and I would very much like to fill it for my own needs, as well as hopefully generate some publicity for my services (filling the gap is enough for me, though)

                I’ve got toys in transit to my LA colocation – two TILEncore-Gx36 cards. And I’m debating whether to get one, three, six, or 12 Odriod C2s for a home arm64/container networking lab. I’ve decided that my decade+ focusing on service provider networks has to come to an end. it is not lucrative, increasingly difficult to find work (that is not mundane or break/fix), and service providers are only becoming bigger (mergers, acquisitions).

                I think I’ll get one, see if it’s what I want, then get two more…and then maybe make an attempt at a variety of arm64 boards for a more “heterogeneous” cluster. though I’m worried I’ll be spending more time getting Linux working on arm64 than anything container networking related!

                [0] http://p4.org/ [1] http://pnda.io/ [2] https://napalm.readthedocs.io/en/latest/

                1. 2

                  TILEncore-Gx36

                  Those look great in specs. I came across something similar looking for DDOS mitigation at 100Gbps. The solution I found used three, Tilera processors in a custom board. The price wasn’t stated. ;) This one you have comes with 36 simple cores, 40Gbps, and onboard crypto for 3 to 4 digits. That has potential for a FOSS attempt at DDOS mitigation or even first firewall the traffic hits. Something to help smaller businesses (including ISP’s) that don’t want to spend six digits. With right hardware and software, cost might go down to a few grand for hardware plus cost of bandwidth.

                  1. 2

                    they do look great in specs, and I look forward to “playing around” with DPDK and NetBSD (if the patches I’ve seen mentioned in old mailing list posts actually exist…haven’t hunted yet). I’m very curious what npf can do with that many cores.

                    The next version, Gx100, are ARM cores. If I could swing it, I’d get my hands on a Silicom 2x100GbE card with the new Intel FM10K chipset.

                2. 5

                  After starting on my rust lock-free B+ tree with log-structured persistence a couple weeks ago, I’ve completed the basics of a lock-free log-structured store and am now building the lock-free pagecache. It’s been great to work on hard database problems again.

                  1. 1

                    Cool!

                  2. 4

                    Not sure. I quit my job and I’ll be joining a new place in March, my last day is early March. I’m thinking I’ll do “annoyance fix fest”, where i tackle a million things that have annoyed me for a long while.

                    Maybe instead i just improve things. I really don’t know.

                    I’m gonna have to learn node for the new gig. Well. Catch up on it anyway. I think it’s going to be fun, and hopefully i can settle nicely there. But you’ll hear about that in this topic in spoke weeks. XD

                    1. 4

                      I have just finished setting up a oVirt/RHEV server out of an old laptop. I’m going to add a couple of other machines I’ve had laying around my apartment to it and make a whole VM cluster.

                      I’ve written up a document on how I’ve done it as it was not obvious. Would my fellow crustaceans be interested? I’m not sure if it would be on-topic or too niche.

                      At $WORK, centralized logging is the goal. But running into various bugs. So debugging the setup this week.

                      1. 4

                        Music and learning synthesis! It’s enormously fun. It’s also rubbing out some rust on my old signals knowledge.

                        I’m also re-reading Bickel and Doksum “Mathematical Statistics, Basic Ideas and Selected Topics, Vol. 1” so that I can get back into the swing of that stuff in order to read Vol. 2.

                        1. 3

                          Visiting Edinburgh, and also the work office as it happens to be here. Internal hackdays, so getting my teeth sunk into Hadoop & Hive as we’ve a need to search our ever-growing logs and keeping them all in Elasticsearch is starting to creak. For longer time periods, pushing that off into a non-instant background search will be a win hopefully.

                          I’ve finally bit the bullet and straightened out the puppet configs for my personal servers, so I can apply them over ssh using the local machine as the puppet master. Also spun up a bunch of modules to configure the SmartOS global zone. Next up is retrofitting puppet to some of the important nodes, and then respinning said nodes afresh.

                          1. 3

                            Making json-to-elm turn JSON, type aliases and decoders into English. E.g “the JSON must have field called name that must be a string”

                            1. 3

                              Planning to write a PCM audio re-sampler in Common Lisp. My video player processes the sound streams already, but a certain system requires one particular sample rate, so transcoding is necessary. The starting point is the basic upscale/decimate + lowpass filter, but if anyone can point it to a bit more optimized, preferably fixed-point algorithm it would be much appreciated.

                              1. 3

                                Last week I did a major new release of our decentralized, post-quantum [1], end-to-end encrypted file system and social network, Peergos (though still alpha) It uses IPFS for storage and P2P transfer, Vue.js for the web interface and GWT to compile all the complex code needed by the web ui to JS. This week I’ll be trying to figure out why it doesn’t work on iphone and Safari.

                                More description. Any comments/feedback/suggestions are very welcome.

                                [1] Currently only quantum computer resistant for files that haven’t been shared with other users and for logins.

                                1. 3

                                  Just started writing a tech book proposal! Also ordered a standing desk to help since I’ve been suffering from RSI in the laptop-at-table configuration.

                                  1. 3

                                    Continued to play with Red-lang. Unfortunately, the Red-C interface is really barebones. For example it can’t represent char arrays in a struct, it can only understand pointers to null-terminated ascii strings.

                                    In order to make progress, I decided to write a small shim in C and compile with the Tiny C Compiler, exporting a function signature that Red can understand. Unfortunately, I seem to have found a bug when compiling x86 code with TCC. When targeting x64 my code works, but Red-lang is 32-bit only for now (on Linux as well as Windows) and can’t load an x64 DLL.

                                    It seems that Red is still too alpha for me to be productive in it, but I’ve started looking at Scheme instead. TinyScheme seems to be an interesting implementation capable of compiling down to ~120kb. At present I’m trying to get its FFI to work.

                                    1. 2

                                      TinyScheme was the one used by that adware guy in the interview posted here recebtly. Where he claimed to have distributed more Scheme than anyone. So, TinyScheme is field-proven in a way. ;)

                                      1. 1

                                        What were you trying to use Red-lang to accomplish?

                                        1. 3

                                          One of my long-time projects is creating a modding tool for editing Halo 1 maps while the game is running (PC platform, not Xbox). The current incarnation is a mix of C# and Python. This works reasonably well, but GUI development seems prone to cruft, and I wanted to see if I could create a reactive GUI written in Red.

                                          In order to read the values from Halo, I need access to the kernel32 APIs ReadProcessMemory and WriteProcessMemory. But even before that, I need to enumerate the processes on the system and search for 'halo.exe', using CreateToolhelp32Snapshot, Process32First, and Process32Next.

                                          I find Halo modding to be a good playground for testing out new technologies, because it’s not trivial but I understand the domain and it’s pretty fun to hack on. Halo maps are really just C structs and pointers… in fact the map files need to be loaded into memory at exactly 0x40440000 in order for the pointers to be valid. If you poke values, the game reflects the change immediately.

                                          1. 3

                                            Seems like something that LuaJIT might be interesting for as well, since it’s ffi is supposed to be very good.

                                            Red looks like it will be very interesting in 3-5 years, but it still looks very alpha.

                                            1. 3

                                              Yes, Lua is the most practical choice, but I’ve been meaning to learn Scheme for a while now. Plus, TinyScheme actually appears to be smaller than Lua… with a dynamically linked msvcrt.dll, it compiles as tiny as 72kb.

                                      2. 2

                                        Specification of a new (libre) ERMS that will focus on projects presented as a Work breakdown structure with machine readable documents as the deliverables. The goal is to digitalize the whole public procurement process and budgeting for at least two Czech agencies and probably a few municipalities as well.

                                        I hate human-invented rules and legislation. Mostly.

                                        1. 2

                                          Due to a component on large school project being postponed by a professor, I was able to get an implementation of eq built into PISC. diff for the curious. I did this in preparation for starting to build a test suite so that I have a better means of handling regressions and the like later if I start doing more optimizing or the like.

                                          I’d like to turn PISC into something for scripting chat-bots similar to how hugo is used, but with less operational complexity, (persistent storage of information without requiring Redis is one idea). I’d also like to build a nice Web IDE for writing said scripts.

                                          Ultimately, it’d be fun for PISC to turn into a nice general purpose language, but I’ve figured that if I can embed it into a nice, cross platform (slack/discord/irc/etc) chat bot, that might help attract the attention of a few people. (If nothing else, it would be fun for the #lobsters or #ludumdare irc channels)

                                          Since I’m in my final semester for a Bachelors of Software Engineering, my focus is elsewhere.

                                          At work, I have to figure out how to write a WordPress plugin, and work out a few kinks in the signup process for a client’s website.

                                          1. 2

                                            Setting up a MEME pipeline in some part or parts of the Google ecosystem. Wishing I had the motivation to rewrite the push2 frontend and add deployd as a service offering in push2.

                                            1. 1

                                              Cool! I’d be curious to hear more of the context for this work!,I have fond memories of using MEME back when I was doing genetics research :)

                                            2. 1

                                              Generally just work, and trying to get back into the swing of working out on a daily-ish basis.

                                              I have a few blog post ideas to work on as well, but I was sick the past few weeks, so I’m still getting over that slightly.

                                              1. 1

                                                Fun: more blog posts about static sites for https://www.aerobatic.com/blog

                                                Not so fun: WordPress theming and consulting for friends that I like, but code that I hate.

                                                1. 1

                                                  I’m cleaning up a Pelican to Wordpress converter - Peliword.

                                                  It works great but needs a refactor and unit tests.

                                                  1. 1

                                                    Writing a serverless landing page in TypeScript+React+Lambda.

                                                    1. 1

                                                      I’m a sophomore in college. I have to write a C++ class called circle that can be used to set a circle radius, calculate area of a circle, calculate circumference of the circle, and return the respective information. Then another class that has a dynamic array of pointers to those circles. I’m waiting to get into the actual fun stuff in my junior and senior year.