Threads for geekingfrog

  1. 14

    I like this point of view. A software artisan has control over how they work. Not everything has to be measured and optimized for efficiency as if your backlog was a conveyor belt.

    1. 5

      Same here - it’s a refreshing point of view to look at developer energy. In corporate terms, morale would be the term.

      1. 4

        But then, you have to measure and optimise corporate morale as well, so that you can grade managers and have an objective metric to evaluate your efforts on that front /s

    1. 4

      After reading their home page description, I have no idea what this is actually for:

      Servo’s mission is to provide an independent, modular, embeddable web engine, which allows developers to deliver content and applications using web standards.

      1. 23

        It’s Mozilla’s effort to write a browser rendering engine in Rust. Started out as a research project, produced some decent components that made it into Firefox, and then got shelved a year or two ago when Mozilla came under new management and decided to cut costs.

        1. 1

          doesn’t it also perform certain threading improvements which you normally don’t have in render engines ? (apart from the CSS layout system already being threaded itself)

          1. 5

            Yes it does. Servo is far more parallelized than traditional rendering engines (both Gecko and KHTML-descended - i.e. WebKit and Blink) because of both Rust’s safe parallelism fu and the absence of decades of legacy code architecture. This is why Stylus and WebRender (Servo components that were ported into Gecko), for example, are so much faster than their predecessors in Gecko.

        2. 10

          This is an engine to render web content. Firefox has gecko, and all the chrome clones use blink. It converts html + css into pixels for your screens.

          1. 4

            I know Tauri has theorized using this as the embeddable web engine (instead of WebKit) with the goal of improving cross-OS consistency. Not sure if this funding news is an any way related to that, though.

            1. 2

              Bundle Size

              By using the OS’s native web renderer, the size of a Tauri app can be less than 600KB.

              I dunno about this. One of the reason Tauri outputs can be so small is the leveraging of the “web view” on the existing platform (as opposed to shipping the entire Chromium engine like Electron). It could help with the consistency, but I would assume this would fall under a flag where some will prefer the lighter footprint over that consistency.

              1. 1

                Agreed — I don’t anticipate it becoming a default. I don’t know what the binary size of Servo is, but hopefully it can be a compelling option against Electron/Chromium since it isn’t a fully fledged browser.

                1. 3

                  Anything to compete against Electron/Chromium/Blink/V8 would be awesome. I wonder if Ladybird is doing well.

                  1. 1

                    At some point, the size of functionally-equivalent C++ and Rust binaries should more or less converge, at least assuming similar compiler backends + codegen (I.e., LLVM/Clang).

                    Choice of coding style, size of external dependencies, asset storage, etc. are all independent of implementation language, but have a sizable impact on binary size.

                    I’m personally more interested in Servo for a) choice and competition against the “New IE”, and b) an engine and community more open to experimentation and extension than Chromium/V8.

                    1. 2

                      At some point, the size of functionally-equivalent C++ and Rust binaries should more or less converge

                      For sure — though is Servo closer to being functionally-equivalent to Chromium, or Blink? I guess my hope is that Tauri + Servo is smaller than Electron + Chromium. From a quick skim it appears Chromium is substantially bigger than Blink itself, so Tauri only requiring the web engine aspect might be a size gain alone.

                      I’m also on board with the added competition, as long as it’s maintained going forward. If a new engine gains usage but then ceases development it would serve to slow adoption of new standards.

            1. 19

              I wouldn’t have deleted that key on their behalf. If it was running some kind of critical service it would now be failing, and services might be at risk, services potentially critical to human life. It’s also Unauthorized Access to a Computer and you shouldn’t trust a corporation to not take legal action against you when it has the opportunity.

              1. 5

                The blog appears to be ran by a British citizen who lives in London, so short of the US govt getting involved, there isn’t likely much Infosys could do, even if they got super duper upset about it.

                US laws do not apply outside of the US, despite the US not always acting like that’s the case.

                That said, I agree it wasn’t the best action they could have done, but hindsight is 20/20 and all.

                1. 2

                  US laws do not apply outside of the US, despite the US not always acting like that’s the case.

                  If you hack into something that’s hosted on US soil, or route traffic across US soil to do it, you can bet US law applies. The only question is whether the country you’re currently in will extradite you.

                  Or, more simply: laws still apply just fine on the internet and you probably rely on that being true, whether you realize it or not.

                  1. 6

                    I completely agree that US laws apply on US soil, obviously they do. They just don’t apply outside the US at all, unless the other countries want them to apply. It’s the treaties and the UK’s willingness that matter here. It’s hard to say how the UK would handle this particular case, assuming the US govt got upset enough to bother the UK about it.

                    My comment that you are quoting was more about: The US govt can generally bully their way into whatever they want in most places on the planet, since they currently have the largest military and economy around.

                    1. 5

                      The current UK prime minister is the son-in-law of the founder of infosys. So I don’t think it would take too much to inflict pain on the author of this blog.

                      1. 3

                        Wow, that’s unfortunate for the OP. Though at the rate the UK is currently going through prime minsters, that may change tomorrow.

                        1. 2

                          My first reaction would be “surely they wouldn’t do anything so petty?” but then I remember who is running the UK at the moment and now I’m not so sure.

                        2. 2

                          Any type of network or equipment that’s on US soil is, well, on US soil. Any sort of entity you affect that’s on US soil is on US soil. Lots of things are actually on US soil.

                          “But the person sending the bytes over the wire wasn’t in the US” doesn’t change that. At best it just means now two countries can each carry out a prosecution, and the person hopes the one they’re currently in won’t do that and won’t extradite.

                          This isn’t some sort of completely new unheard-of never-before-considered untested thing, either. Extradition treaties, and other procedures for handling people who think they’ll evade punishment by being on the other side of a border, is something that literally goes back millennia.

                          1. 1

                            The only part I disagree with is: “At best it just means now two countries can each carry out a prosecution”.

                            This assumes the action is illegal in both countries. In this case, where the OP deleted the AWS key, that’s possible, but I wouldn’t say it’s certain. That’s for lawyers to fight over, if it ever gets that far.

                        3. 4

                          US law does not apply outside the US, some Americans just think it does.

                          1. 3

                            If what you do passes through wires, networks, servers, routers, anything on US soil, then it was not “outside the US”.

                            Like I said to the other person: you probably, whether you realize/like it or not, rely on the fact that wherever you reside can in fact enforce its laws in this fashion, regardless of which country you reside in.

                            1. 2

                              If this comes as a surprise to anyone, consider the story of CSE TransTel, a telecom company, and its parent company CSE Global Limited, both based in Singapore. CSE TransTel signed a contract to install communications equipment inside Iran, and paid purchase orders to Iranian companies to support delivery & installation of their equipment. They made their payments out of a Singapore-based bank.

                              What’s the problem, you ask? They made payments out of an account denominated in US dollars. These payments were processed through the US financial system: as a result, the US government argued that the actions of an entirely foreign company using entirely foreign banks resulted in financial institutions in the US handling payments to Iranian companies, which violates sanctions against Iran. This created a US nexus that made otherwise totally legal actions impermissible under US laws.

                              CSE TransTel settled with OFAC for twelve million dollars. Why? They’re based in Singapore?! If they didn’t, they’d end up listed as a specially designated national and any US company or person would be legally barred from working with them or risk OFAC sanctions of their own.

                              The US legal system and enforcement regimes will take a very broad determination of jurisdiction, and any company – web hosting, infrastructure, payments – with a US connection are legally required to fall in line.

                              1. 2

                                From my other comment: The US govt can generally bully their way into whatever they want in most places on the planet, since they currently have the largest military and economy around.

                                Here CSE TransTel had to have known it was a bad idea to sell to Iran, since even their own government is less than pleased with Iran’s nuclear weapons program. They probably thought about it, and figured it was worth trying, got caught and eventually gave in, knowing their own govt wasn’t really on their side either.

                                I’m not necessarily against the US Govt’s bullying tactics, it helps the world just get stuff done sometimes, but it is a power they can(and arguably have) over-used sometimes.

                                1. 1

                                  You seem to have a very specific political axe to grind, but it’s not applicable here.

                                  To see why, imagine there’s a building near an international border, and someone on the other side of the border throws a rock across and breaks a window in the building. The country the building was in can call it a violation of their laws, even though the person who threw the rock wasn’t on their soil. Whether the person who threw the rock will actually be punished by the country the building was in depends on the existence and details of extradition treaties, but nobody should be surprised if that person gets extradited to face consequences in the country where the building was.

                                  The internet didn’t change anything about this. If you send bits over wires, and some of those wires are in another country, that country’s laws apply. It’s not “bullying” or some sort of new, unique, just-made-up recent thing. Like I already said in another reply, we’re talking about things that political and legal systems have been dealing with for literally thousands of years at this point. Rather: a lot of people hoped and wished and wanted the internet to somehow provide a new, never-before-seen type of extraterritorial place where those political and legal systems couldn’t reach, but their wanting and wishing didn’t and hasn’t made it so. Instead, long-existing frameworks have been adapted as needed, and that’s that.

                                  1. 1

                                    You seem to have a very specific political axe to grind, but it’s not applicable here.

                                    no? You seem to be misunderstanding what I’m saying perhaps? I’m a little confused by this comment.

                                    Anyways, The US and the UK have an extradition treaty, and the UK government is happy to publish it here: https://www.gov.uk/government/publications/extradition-treaty-between-the-uk-and-the-usa-with-exchange-of-notes

                                    I’m not currently an international lawyer and I haven’t read the whole thing, but skimming through it, it seems to say, In general, if it’s against the law in both countries, then they will automatically extradite people either direction. Which seems totally reasonable to me.

                                    Nowhere in there does it say that US laws apply in the UK, as that is straight up ridiculous. An easy example of how ridiculous that is: Guns are generally illegal in the UK and are generally not illegal in the US.

                                    1. 1

                                      You seem to be misunderstanding what I’m saying perhaps?

                                      Over and over you single out one and only one country and talk about “bullying”.

                                      Nowhere in there does it say that US laws apply in the UK, as that is straight up ridiculous.

                                      The issue here is you are the one who is trying to argue that this is somehow “US law applying in the UK”. Not me.

                                      I’ve explained to you multiple times now that it is an extremely normal and banal and accepted and uncontroversial idea that you can break the law of a country by committing acts that involve or have effect on entities or infrastructure in that country, even if your physical body was not physically within that country’s borders at the time.

                                      But this is not the same as saying a particular country’s laws apply everywhere – thus the example of throwing a rock over the border and causing damage on the other side, which hopefully is a pretty clear and common-sense example of the underlying principle.

                                      1. 1

                                        Over and over you single out one and only one country and talk about “bullying”.

                                        Would s/bullying/interfering/g be a better word for you? The US is far from the only ones that do this type of behaviour. Generally it’s larger countries relative to smaller countries, that the US is the largest just makes them more effective at it.

                                        The issue here is you are the one who is trying to argue that this is somehow “US law applying in the UK”. Not me.

                                        Then I apologize for my part in our miscommunication. Though I find it very confusing that you think my position is that US law applies in the UK. Clearly we don’t seem to be communicating well during this course of conversation. With such gross miscommunication, it’s probably easier to just stop. Especially since the stakes for you and me are at worst some feelings being hurt. Have a pleasant and wonderful weekend!

                      2. 4

                        I mean, it’s sketchy, but it does seem to be a key used for development, and which had been inactive for a whole year. Granted, anyone who screws up by issuing AdministratorAccess keys to individual developers might also run some critical service under them, but given the context (running some statistical models over externally-hosted records from several sources) it appears rather unlikely that it was used to run anything critical to human life. The key was, after all, used by Infosys to run things at their end, not by JH.

                        I don’t wanna defend what the author did, I’m, not sure I would’ve done it that way, either, but I do think it was quite safe to do from a technical standpoint. From a legal standpoint, based on my experience working with (and, sadly for my mental sanity, occasionally in) outsourcing companies, I doubt there is anyone at Infosys’ end who can a) read logs and b) is not on the verge of ragequitting, so there’s probably no one to notify the Legal team about it :-).

                        1. 6

                          It might seem that way, but there was no way for the author to know. They should have reported to infosys and Johns Hopkins.

                          As it is, the author has potentially harmed people and/or incurred liability.

                          1. 13

                            It seems like the author ended up doing that precisely because they couldn’t contact either JH or Infosys. There’s obviously no way to verify that, but I have been at the receiving end of the problem. Someone went public with several issues in a program that the company I was working for sold. The higher-ups got very butthurt, nasty press release came out…

                            …turned out the researcher had tried to contact them through several separate channels, but messages got ignored each time because they weren’t read by anyone who actually understood what was being said to them. One of the official channels for reporting security issues was mostly unused, because people usually went through unofficial channels. IIRC the people who supposedly monitored that channel weren’t even working there anymore. Dude ended up going public because he thought it was likely the only way to actually prevent anyone from getting harmed, despite incurring liability.

                            1. 1

                              Were there any legal consequences?

                              1. 3

                                AFAIK no, and the whole thing was dropped like a very hot potato the moment people realized there had been as much as one attempt at responsible disclosure. I mean it’s not 1992, companies are legitimately expected to make this no more complicated than a couple of Google searches and an email.

                                Management is rarely inclined to litigate when there’s a looming PR disaster in it. A lawsuit moves slowly, even when coaxed with money and connections, whereas social media and the press operate on an hourly timetable. Realistically, there’s barely anything to gain from a lawsuit on a matter like this, and potentially a lot to lose in terms of PR and community relations – they only move forward if someone in the legal team really needs to prove themselves. Even the financial incentives are practically zero, the kind of sum they could get is probably in the sort of amount that companies like Infosys regularly write off for government bribes.

                                1. 2

                                  That’s my view as well. Infosys would be very stupid to raise a legal stink about this, as it would shine a light at their alleged incompetence at deploying code and responding to disclosures.

                            2. 3

                              You’re right, but the flip side is reporting it properly, having them not do anything about it, and then a bad actor finds and uses it. Not much to recommend one over the other imo.

                              1. 8

                                From what I’ve seen, you may run into careless business associates / sub-associates, but covered entities are often very wary of the risk around HIPAA violations. It sounded like the author attempted to report to Infosys directly so I’m not surprised he hit a wall.

                                So again, if you find PHI – "Johns Hopkins Hospital" "general counsel" into your favorite search engine took me straight to their legal department, including direct contacts to HIPAA lawyers. Even without specialist lawyers, just get in touch with someone in their legal / leadership chain. The magic happens when you say “I’d like to report a HIPAA violation” to a human, preferably a human on a legal team.

                                And if you truly can’t get anyone to act, HHS has a process to report complaints directly to them. It’ll likely take longer for them to act, but they have broad leeway to sanction bad actors and will get the attention of the offender.

                                1. 1

                                  On the other hand, people not living in USA might not be be so intimately familiar with USA laws and compliance culture.

                          2. 1

                            All access to remote computers is unauthorized. Maybe we should stop allowing corporations to hurt themselves and others, even if it means violating their privacy.

                          1. 13

                            I find readability is largely something that comes out of familiarity. S-expression based language like the lisps look completely alien, unless that’s what you’re using daily and then the c/java/js syntax is the one looking weird and clunky.

                            Overall, I find myself paying less and less attention to syntax as the years go, and more about the semantics of the languages.

                            One exception are auto formatting tools, which I really appreciate, as it frees us from tedious manual formatting or useless debates.

                            1. 2

                              To a degree I agree, but I believe there are objective claims we can claim about the cognitive load of different syntaxes. I think syntax & semantics are also closely tied, syntax enables/disables semantics in many cases.

                              If your syntax only allows for positional arguments its very easy to argue thats less readable than a language like jakt which requires named arguments to (nearly) every function & constructor.

                              I also think the syntax of the language standard api is part of this convo too. Readability is not the same as “easy to reason about”, its related but distinct, id argue its specifically the amount of cognitive load it takes to be satisfied with understanding an isolated function.

                              That means understanding the basic language semantics, argument order, inferring data structures, keeping track of intermediate values (which gets duplicated x times with recursion). These and more are the trade offs, we can obviously get very skilled at these things but it doesn’t mean all languages are essentially the same.

                              1. 1

                                I agree, I keep wanting to post a parody comment showing a “clearly more readable version” in APL.

                              1. 4

                                That looks pretty cool. How does the power consumption look like? Many years ago I checked how google was handling its notification, and it was pretty involved, with some help from the carriers to minimize the power consumption. Is it still a concern?

                                1. 2

                                  If I may quote from the FAQ (https://ntfy.sh/docs/faq/#how-much-battery-does-the-android-app-use):

                                  If you use the ntfy.sh server, and you don’t use the instant delivery feature, the Android/iOS app uses no additional battery, since Firebase Cloud Messaging (FCM) is used. If you use your own server, or you use instant delivery (Android only), the app has to maintain a constant connection to the server, which consumes about 0-1% of battery in 17h of use (on my phone). There has been a ton of testing and improvement around this. I think it’s pretty decent now.

                                  Happy to answer other questions you have.

                                  1. 2

                                    Oh didn’t see the faq, thanks, that’s pretty good to see that. Even the version not using fcm is good.

                                1. 10
                                  1. 6

                                    And even more QR code tutorial: https://www.thonky.com/qr-code-tutorial/

                                    1. 1

                                      The idea of strapping a QR code onto an airplane is fascinating. I wonder if it was readable from a phone on the ground, with wind blowing it around and the perspective distortion..?

                                    1. 3

                                      Cool talk. Are there a bit more details about the jsonfeed automation thingy? That seems interesting but I don’t know how this works. Did you also build the tools to read and act upon new stuff there? 3rd party? Something else?

                                      1. 2

                                        I built it myself. I’ll write an article on it after I rewrite part of it to be more modern. I made a mistake with its design and need to rethink a lot of it.

                                        1. 2

                                          Cool, looking forward to it. I’m also in the process of RiiR my site, with a somewhat similar design, and I want to add “companions” like you and cool bear from fasterthanlime. Your explanation of what you did was helpful. I was planning of doing markdown extension, but I’m going to skip that now.

                                      1. 3

                                        I had a similar idea with aws nfs as a service, never tried it but expecting similar results.

                                        1. 1

                                          Notably, Google Drive does not allow you to partially modify a file: you can only do whole replacement (similar to s3).

                                          Practically, using a nbd (Network Block Device) is going to be a more direct way to accomplish this goal.

                                        1. 1

                                          I’d love to see a video of this system in action. Fantastic project, the world needs more of these wacky things, it’s a lot more fun.

                                          1. 5

                                            Regarding the deadlock at the end, besides miri that may not be suitable for application code, are there other tools like linters or runtime check to be able to detect and debug these problems?

                                            edit apparently there’s at least an experimental deadlock detector in the parking_lot library. Question still stands for other usecases.

                                            1. 1

                                              I wonder what the best way to provide the author with proofreading feedback is.

                                              1. 1

                                                email works fine. Comments there do also work.

                                                1. 1

                                                  I didn’t see a place to leave comments there but will send email.

                                              1. 3

                                                At multiple jobs now I’ve changed people’s lives (only mildly exaggerating) by showing them things that I’d picked up sort of by osmosis and long took for granted: history searching in the shell, readline keybindings to quickly navigate/edit lines in a shell or in many REPLs, and a few common tips and tricks for navigating from the command line and creating/moving files and directories.

                                                At one company I got to write a long-ish thing that became part of the engineering team’s documentation, trying to do an intro to productive use of command-line tools. I wish more companies would invest in and allow that, because the gains can be significant.

                                                Some iTerm2-specific tips I’ll add on here for fun:

                                                • Cmd+Shift+H will pop open a menu of strings you’ve recently pasted into the terminal. So if you’ve pasted something that isn’t a command and since put something else in your clipboard, or if it was a command but you don’t remember enough of it for an efficient history search, you can use that to try to find it again.
                                                • Cmd+Option+B opens a slider to let you replay the history of what was visible in your terminal, useful for programs with output that disappears when they exit.
                                                • Cmd+Shift+E will add timestamps to each currently visible line, telling you when they were output to the terminal, useful for seeing when something happened in a longer-running process.
                                                1. 1

                                                  That sound super useful, is there a link to this doc? I had something similar in mind when I wrote that, inspired from my interactions with coworkers who aren’t comfortable at all with the terminal.

                                                1. 4

                                                  There’s a lot of unwrapping in the code, and I wonder if setting the panic strategy to abort would be useful for also making the resulting code leaner. Though we’re talking diminishing returns when it’s already pretty fast, it may be fun to measure.

                                                  1. 2

                                                    Interesting, I wasn’t aware of this option. Although it seems it’s only in nightly. I did try it, but the difference was about 3 or 4%. It’s something, but not massive either.

                                                    1. 5

                                                      Although it seems it’s only in nightly.

                                                      panic=abort in Cargo.toml is fine on stable Rust.

                                                      1. 1

                                                        Oh nice, I didn’t know about it.

                                                  1. 2

                                                    Awesome! Great results. Thanks for linking my 2020 post as well.

                                                    This year I’m down to 50ms, though I still have to finish the last two days. Sadly I’m busy. https://github.com/timvisee/advent-of-code-2021

                                                    Funnily enough I did get better results with Dijkstra for day 15. I wonder which is better.

                                                    1. 3

                                                      woha, 50ms is very impressive, well done.

                                                    1. 9

                                                      Interesting comment from Peter Norvig there: http://www.loper-os.org/?p=69#comment-17594

                                                      There are other languages which allow developers to be very productive, and all it takes is spending a bit of time learning them. Having said that, I do agree with the thesis that corporation value fungibility of developers a lot, and I find that soul crushing.

                                                      1. 8

                                                        The response to that by the website’s primary author is as classy as usual, too. With friends like these…

                                                        1. 6

                                                          I think “fungibility” is a particularly pessimistic view. The same benefits that facilitate “fungibility” in a corporate context also facilitate collaboration generally including in the open source, research, and just-for-fun worlds.

                                                          1. 4

                                                            Counterpoint: If you work on something truly valuable to society, it’s irresponsible to make it depend on you alone. We humans are fragile creature, after all.

                                                            1. 3

                                                              Agree, although this is an extreme example. There aren’t that many things that are truly valuable to society. Most of the time, it’s valuable only for your corporate overlord as a mean to extract money, and if that goes up in flame, I won’t feel bad at all.

                                                          1. 2

                                                            Excellent talk. It’s great to see alternative design and novel (?) approaches to OS design. I did enjoyed the part about the nice properties of synchronicity, and the drawbacks of asynchronous systems, which are all the modern rage.

                                                            1. 5

                                                              Several aspects remind me of Separation Kernels, a design from 1973 or so. See muen.sk for a modern take

                                                              1. 2

                                                                I have not seen Separation Kernels before. Thanks for sharing that.

                                                                After a quick look, I would like your opinion. What is the difference between a Separation Kernel and a hypervisor?

                                                                1. 9

                                                                  Hypervisors are generally built for maximizing resource utilization while Separation Kernels assign fixed resources to VMs even if they don’t use them if that helps isolate them from one another, so that even two cooperating VMs can’t communicate (e.g. through side-channels) unless the SK explicitly allows it as part of the active policy.

                                                                  Memory ballooning, shared NIC bandwidth, shared I/O bandwidth, CPU cache behavior, various properties around GPU resources can be abused as a side-channel (even if very low bandwidth - although once identified, researchers usually manage to increase that considerably) but all of that still makes sense for “normal” hypervisor setups.

                                                                  (Correction to my earlier post: Separation Kernels were discussed in 1981. The more general Security Kernel concept was discussed in 1975 and Bell & LaPadula “Secure Computer System: A Mathematical Model” appeared in 1973. I haven’t been very active in that space for quite a few years so I mixed up the timeline.)

                                                                  1. 1

                                                                    This is excellent. Thank you.

                                                            1. 3

                                                              Interesting read. I would like to see the how they found out the bottleneck, I find that at least as interesting as the optimisation in itself.

                                                              1. 27

                                                                Interesting article, but I disagree with the conclusion. /events and webhooks have very different tradeoffs, one is not just superior to the other. As stated at the beginning, webhooks are there to avoid polling. Even with long polling, this means your servers have to maintain a lot of connections open, the burden is on the provider. Webhooks shift that burden on the consumers, and as noted it comes with its own set of drawbacks.

                                                                I think the solution you want to go for really depend on your usecase and what you are willing to pay and code for.

                                                                1. 10

                                                                  Yeah, long polling/websockets feel like they are ineffective in a big-O way here. Without thinking too much, it seems like the sweet spot is using /events as a source of truth, and providing a single, best-effort hook, “something’s changed, please re-poll /events”.

                                                                  1. 10

                                                                    it seems like the sweet spot is using /events as a source of truth, and providing a single, best-effort hook, “something’s changed, please re-poll /events”.

                                                                    That’s an option suggested in the last few paragraphs of the article.

                                                                    I think it’s a “sweet spot” for a free service, but it’s neigh impossible to offer any kind of reasonable SLA without overbuilding so I do not recommend it ever.

                                                                    long polling/websockets feel like they are ineffective in a big-O way here

                                                                    Feelings can be confusing sometimes.

                                                                    Polling /events without long-polling means it’s going to be customer-controlled. You may be able to influence this at your load-balancer.

                                                                    In both polling /events and long-polling, the best-case RTO is about 3x RTT, but the worst-case differs greatly: With polling, it’s the frequency of your customer-controlled poll, whereas for long-polling it’s about 3x RTT (in an ideal implementation[1], and a trivial one[2] can get 3 seconds no problem).

                                                                    Long-polling has another advantage: Simply route users to the same machine. This makes capacity planning and failure-detection trivial (server simply disconnects old pollers).

                                                                    An open but idle TCP connection takes around 2k per. This can be amortised across multiple machines. The capacity can be planned in-advance, so errors (and retries!) visible to the client are uncommon. Polling is capped by a single server setting, and errors (retries!) are visible to clients. Without careful configuration, a misbehaving client can trivially deny service to legitimate clients.

                                                                    In general, once you weigh everything: long-polling for a fixed set of clients provides the best experience to both producer and consumer, with the fewest number of failure conditions to handle, with a fixed memory cost that can be precomputed. If you’re doing something for free, and so you don’t care about your consumers (i.e. they can’t email you), you can just have people poll, because it is slightly better for the producer.

                                                                    [1]: If you know the RTT, both sides can set TCP_USER_TIMEOUT to the distance (I like to add 10% or so for fuzz) when polling and enable quickacks while waiting. When sending data, simply turn off this option and turn on NDELAY. I do this in my SDKs, but if I have to publish a REST-style API I find supporting/explaining socket options to random people exhausting.

                                                                    [2]: Simply turn on TCP keepalives at both ends, and set the interval timer to the lowest.

                                                                    1. 1

                                                                      Thanks for elaborating!

                                                                1. 1

                                                                  Good guides, learnt quite a bit (with the other pages as well). I also appreciate to have the vim help reference, they are usually quite good and too often missing from vim guides/tutorials.

                                                                  1. 9

                                                                    You can now use a pyproject.toml file instead of mypy.ini for mypy configuration.

                                                                    I’m excited about this one. I was recently introduced to poetry and it’s been a breath of fresh air for managing dependencies in python. Poetry uses pyproject.toml for all its configuration, so being able to kill off other config files in the repo will be nice. After mypy, I’ll just need to wait for a version of pytest that does the same, and then I’ll be set.

                                                                    1. 3

                                                                      Agree, it’s really nice to see multiple converge toward this one config file. Black and isort can also be configured through pyproject.toml