1. 7
  1. 4

    Good advice! If you’re implementing one of these, I’d suggest checking out asyncio.Protocol and related classes. They can handle various common forms of I/O in a standardized way, which makes it easier to decouple protocol logic. The article mentions asyncio of course, but I’m not sure that the Protocol classes were in the standard library in 2016, when it was written.

    edit: I suggested ‘python’ tag to replace ‘programming’. Yes you can do this in “any” language, but the post is all about how to do it in Python.

    1. 1

      edit: I suggested ‘python’ tag to replace ‘programming’. Yes you can do this in “any” language, but the post is all about how to do it in Python.

      Interestingly, I can’t edit or suggest flags on my own post. I’m not too unhappy with the ‘programming’ tag, though: to mirror your sentence, the links are all to Python libraries, but the article applies to any language.

    2. 2

      I found wring I/O-free code pretty easy for my DNS library—it provides two functions, one to encode a DNS request, and one to decode a DNS request, so it’s easy to drop into any network framework (it’s been used at work for a few projects). I do wish the author provided a bit more details about TCP based protocols because those a bit harder to deal with. I was able to implement an API that works for both blocking and event driven programming (for both raw TCP and TLS) but it took a bit of work (the event driven stuff only works for POSIX).

      1. 1

        So I have zilch experience with implementing network protocols myself, but I’m curious: what makes it harder to implement the logic of protocols that work over TCP? Alternatively, what did you run into during your own project that a next person could have an easier time with if only they know? Pure curiosity on my part, don’t feel obliged to write this up,

        I had a look at the list of sans-I/O implementations on the main page for you, but there is no TCP library among them, if that’s what you were looking for. It does contain sans-I/O HTTP/1.1 and HTTP/2 libraries, though, which both have some mentions of TCP in their examples.

        1. 2

          Most of the protocols on that list are over TCP (or over TLS which is over TCP). The issue I had with TCP is its streaming nature. With UDP, you get the entire packet to work with, but with TCP, you may have to wait for more data to stream in. For protocols which have lengths defined, this is pretty easy, but for text-based protocols (like SMTP or HTTP/1) you have to be aware of the boundary conditions and check for them (this gets “real fun” for an event driven program [1]). libtls (which comes with LibreSSL) will report back when it wants input or output, but this can happen for both reading and writing so you have to have a way of handling that.

          [1] This can lead to certain types of denial attacks, where an attacker will connect then feed the protocol a byte at a time very slowly.

          1. 1

            Ah yes, that would be tricky to implement correctly. I’ve just done a closer skim of the h11 and hyper-h2 projects with this in mind, and it appears that they model getting each header/body/status code/… all at once, rather than modelling receiving an unknown number of bytes at time and needing parse them as part of driving the state machine. So no help there, either.

            Thanks for writing that comment, it was very interesting to me.

      2. -1

        Trust Python developers to add another level of abstraction and call it a new programming paradigm.

        Not threading/littering your code with IO calls is no different to recommending don’t write a ball of mud and a natural continuation of group similar things together.

        Claiming IO-less is any different to just creating a class or bit banging via a function is ludicrous. You still have to deal with the problem, this has just moved it elsewhere and tends to let you kick the can down the road in postponing working on the harder part to your problem.

        The rest of the world calls this message passing/actor model (Erlang, F#, …) or channels/coroutines (Go or libdill). Amusing is the irony that a Python developer would propose using callbacks/events as if JavaScript is not something they scoff at or a car crash we should have all learnt from.

        I really don’t understand why developers when they see something fundamental to another language that helps them better their solution, the response is to try and crowbar it into their ‘one true language’ (which ain’t looking so one or true right now) rather than just embrace some languages are better in some problem spaces.

        It is okay to blend languages together, I personally find this an approach that works well for me and the teams I work with. Maybe some would claim ‘we are a Python shop’ whilst ignoring what they are trying to introduce is something their team will struggle with and find themselves fighting with the language to use…unless of course we use another layer of abstraction.

        1. 1

          Dude, that’s not an acceptable way to express advice or criticism. Its phrasing is very contemptuous, and amusingly it assumes that Python developers have the same culture of contempt, but aimed at JavaScript. You might be interested in reading Shaw’s short blog post on Contempt Culture?

          I do assume it was just a slip-up, though – the rest of your posts aren’t like this at all. But, ya. You might want to edit or delete this one.

          1. 0

            I was hoping to start a discussion around not my two cheap pot shots (50 of 250 words) at the Python community, but it seems that is what I have been given to work with.

            The bulk, and overlooked, part of my comment was attempting to flag that the article is void of content but is describing as if it is a big deal. The article reads as if the ideas in there materialised straight out of the ether, which whilst tragic, given the demographic I consider this not wholly surprising.

            When I use my tar brush, I probably should have thrown in the disclaimer, Python is as equally bad as Elixir, Go, Rust, Ruby, … and by the way all languages suck, but some suck less. I see those communities yet just putting a new coat of paint on everything they touch, and pretend it is the next best thing, but worse still is the passive acceptance of it all.

            Sure, I could have dialled back the bluntness say by 30% but my statement is based on my experiences and that is important. Sorry it offends but I am equally offended by the acceptance and unchallenged stories and comments around these parts of “should be written in Rust”, “array languages, perl, … are terrible as it looks like line noise”, “protobuf is great, what is this asn1 cruft” and “using Zoom kills kittens”.

            In the past when I have tried to challenge a different view, @tedu amusingly and correctly summarised that lobste.rs literally considers me the Stasi. Other times I sigh when I see zero love for using OAuth2 implicit grants and refresh tokens safely in SPAs whilst nonsense like an IP subnet calculator bubble up; though I’m willing to accept that this is all pure bitterness on my part. :)

            I’m disappointed that the focus ended up being on the 20% rather than the 80%, and again unfortunately around these parts it seems a prerequisite that the snark needs to be dialed to 0% otherwise it is deemed “unaccepted” and not worth engaging.

            I hate all developers equally if that makes you feel better? :)