1. 6

    IMO a honking great mistake to turn on Strict-Transport-Security by default with a max-age of 2 years. People will drop this in and break their systems and that makes this a footgun. No one should deploy HSTS like that that seems to me to be the consensus among HSTS advocates.

    And why is HTTP caching getting disabled? What does that have to do with security for a website?

    1. 1

      And why is HTTP caching getting disabled? What does that have to do with security for a website?

      it shouldn’t be disabled for all routes, but it minimizes data exposure (1, 2). Basically, if you have anything cached by proxies or in browser, an attacker with something like Local access can steal that data (3).

      1. 2

        X-Frame-Options: SAMEORIGIN is a sane default. Mucking around with cache settings in the abstract with no particular threat model in mind just feels like random, uninformed changes to me.

        How many sites/apis have you personally worked on where there was been serious consideration about how to defend a users local HTTP cache from an attacker on the same machine? I think for me the number is 1, in 10 years, and we weren’t outsourcing it to a third party library.

        1. 1

          How many sites/apis have you personally worked on where there was been serious consideration about how to defend a users local HTTP cache from an attacker on the same machine?

          There are two things to consider really:

          1. that caching isn’t just about the local machine, since depending on the cache control instructions and the path to the end user, it could be cached by other caches, such as proxies
          2. that you only consider the risk of the data being exposed, not that we are defending users from all attacks with Local position

          However, I would say quite a few, esp in the mobile space; lots of mobile applications used to cache all sorts of sensitive data, which could then be exposed by backups or the like. Is this a huge concern for security teams? Generally no, nor should it be; an attacker would need sufficient position, &c. to actually impact incorrect caching, so the Likelihood is low or even very low. That still has worked out to a few hundred reports in my career.

          On the flip side, the Impact could be High or Very High based on the data cached; I’ve seen everything from PANs to user’s PII or other sensitive data to passwords. Again, it depends on the sensitivity of the data being cached, and should be applied selectively. It’s not something you can just scan and hand over, but rather something you work with a system’s actual data to determine.

          Lastly, remember that browser Cache Control nuance led Twitter to have to disclose a DM privacy exposure bug just last year.

    1. 13

      I think I agree with the first person who wrote him a letter. There is a difference between finding more novel and varied examples and picking examples designed to goad your readers.

      Please in the future, remember that we, the book buyers, are looking for information about using PL/SQL. I am as tired of the emp and dept tables as you are, but less distracting examples would have been more appropriate.

      Everyone has a political view and sometimes that arises legitimately in technology but I think it’s just basic self-control to express your political view only where it really might help something.

      1. 20

        The dude’s point is that we all have a political perspective, and we’re expressing it, either explicitly or implicitly. He chose to express his explicitly through the examples in his textbook.

        If you write a database text and fill it with department / employee type examples, shopping examples, and so forth, then you are implicitly promoting a capitalist world view, the same world view that does not bat an eye when using degrading terms like “human resources”. At least here in the US, this sort of thing goes unquestioned, because of the dominant ideology.

        1. 4

          Yes, it’s implicit, it’s unquestioned and nobody bats an eye - and that’s why it makes for better examples.

          Examples require the use of social territory. That territory can be either unquestioned good or questioned territory. When choosing examples in questioned territory, you engage in active cultural participation; when choosing examples in unquestioned territory, you engage in passive cultural participation. Examples should engage in passive participation, because that way they are relatable to the greatest number of readers.

          (You can also use unquestioned bad territory, such as defining a database schema to count Jews in the Holocaust for the Nazis, but then nobody will buy your book.)

          1. 9

            I don’t see why “nobody bats an eye” is a desirable quality for examples or why “active cultural participation” is a bad thing.

            It’s not at all clear to me that the examples given are not relatable or that “relatable to the greatest number of readers” should even be a core value. Perhaps provocative examples engage readers more and cause them to think about the examples more.

            1. 4

              Would be curious how you’d feel if it were something sorting countries by iq or something.

              Would you be happy to be engaged, or be distracted by a thinking about testing methodology and things like that?

              1. 3

                I’d have to see it in context to find out how I’d react. IQ is strongly related to class and similarity to the people who devised the test, and such a table might be part of a demonstration of that.

                Certainly if an example just seemed pointlessly offensive I would think less of the author and maybe choose a different textbook.

                But I think equating a hypothetical very racist example with some examples that are a bit left of centre in the USA is unfair.

                1. 2

                  A substantial amount of political dispute in the English speaking world is precisely about what speech counts as racist and therefore legitimately stigmatizable. Using data that implies that cognitive capacity is meaningfully different between different countries of the world in a programming example constitutes a political assertion that this idea is not stigmatizable; in the same way that the article’s example about a war criminal database constitutes a political assertion about how people should see Henry Kissinger.

            2. 6

              But now you’ve thought about it, so it has become active participation. From now on you are obliged to make sure your examples completely apolitical.

              Consider engineers have a code of ethics, https://en.wikipedia.org/wiki/Order_of_the_Engineer

              If your work includes producing examples they should “serve humanity”. I cannot conscientiously make examples that promote capitalism, but giving examples that might make people think about world affairs would be okay.

              1. 3

                Yes, it’s implicit, it’s unquestioned and nobody bats an eye - and that’s why it makes for better examples.

                That assumes a lot from the readership. For a mundane, apolitical example, I submit children to this discussion. For most my childhood due to various reasons, I only had access to a Pentium. It didn’t have a network connection, and I eventually installed Linux on it. Because Linux made it so easy to code, I would try to check out books from the library and learn how to write code, but all the examples were completely unrelatable to me as a pre-teen. Employee this, business that, I realized even at the time that the examples were meant to be highly relatable to practitioners, but I honestly found math much more interesting than these soulless books because I was unable to relate to them in any way. That was one of the big reasons I started out coding by trying to write games; game programming books felt much more relatable to me as a kid who read a lot of books and played video games than these soulless books about employee hierarchies and recipes.

                Also, it’s important to keep in mind that the conditions that make something unpolitical are pretty restricted in context. Someone growing up in a developing country or a country with a very different economic ideology will probably find these staid business examples just as unrelatable as children. International editions of textbooks frequently do change examples for exactly this reason.

            3. 5

              Everyone has a political view and sometimes that arises legitimately in technology but I think it’s just basic self-control to express your political view only where it really might help something.

              I am totally with you on this, and do my best to keep my political perspectives away from the technology work I do as much as I can. I have worked on projects with ethical/political considerations (whether someone might consider a few of these projects ethical depends on their personal political leanings.) Definitely a touchy subject.

              That being said, I have a really hard time empathizing with the readers who wrote in to complain that the examples are too distracting. I believe a database book aught to have concrete examples while teaching the abstract concepts (e.g. it’s a book about writing databases in general, not “how to keep track of war criminals”). My own personal reaction to the examples talked about are “ok, whether I agree with the premise or not, these examples have interesting abstract concepts that they’re illustrating.” There are lots of systems that exist in this world whose existence I fundamentally disagree with, but where I’d also love to pop the hood and figure out how they work!

              In fact, as I sat here thinking about this, I started wondering if, for me, this style of examples might actually help cement specific concepts with easy mental look-up keys; I can imagine coming to a database design problem and thinking “oh, this is like the Kissinger problem.”

            1. 6

              I use UNIX sockets whenever I can. Using TCP/IP for connections to local programs seems so excessive and is involving a lot more machinery and overhead. I’ve even seen people use SSL by mistake for local connections when using templating config managers.

              Most common server programs support UNIX sockets: SQL databases, memcache, pythons wsgi servers, nginx, Apache, etc. Maybe the most notable exception is rabbitmq? I just think people don’t know they exist, or find them mysterious.

              1. 2

                You can, of course, use socat as a tcp proxy to a Unix domain socket. You’ll lose the performance benefits of UDS but can interact with TCP-only services without needing to pull in a network stack into your application.

                1. 2

                  Using TCP/IP for connections to local programs seems so excessive and is involving a lot more machinery and overhead.

                  Both TCP/IP and UNIX sockets are abstracted away by the kernel in my mental model. Where can I read more about their overheads?

                  1. 2

                    The main thing that comes to mind is this (postgres) comparison:

                    https://momjian.us/main/blogs/pgblog/2012.html#June_6_2012

                    I have done a number of private benchmarks over the years which find about the same but it is sometimes very obvious because on debian at least the default postgres connection is a unix socket and when you start using TCP/IP instead (eg when using docker for integration tests) some applications can slow down a bit - particularly noticeable on test suites that output timing numbers.

                    1. 1

                      Thanks for the pointer! That’s a good read, but I want to understand the overheads from a theoretical perspective, like, which steps are handled under the hood by the kernel when I use a UNIX/TCP socket?

                      1. 2

                        My own (perhaps naive) mental model is that the TCP/IP socket is approximately all the work described in my undergrad TCP/IP textbook:

                        • copy into a TCP frame
                        • copy into an IP packet (though I know these two steps are clubbed together in practice)
                        • figure out where to send the IP packet - easy as it’s localhost
                        • pulling apart the IP packet
                        • pulling apart the copied TCP frame (again, clubbed together normally)

                        as opposed to a unix socket which is basically two files, one in each “direction”. And on unix a file is “just” a seekable stream of bytes.

                        I suppose if I wanted to know exactly which steps are in userland vs which are in the kernel I would review the kernel syscalls that my fave language’s implementation uses.

                        My ideas for intro reading (ie the books I liked):

                        • For networks, a) Tenenbaum’s Computer Networks or b) TCP/IP Illustrated vol 1
                        • For files, the relevant bits of a) Tenenbaum’s Operating Systems or b) the Operating Systems dinosaur book

                        Two books I want to read are Robert Love’s Linux Kernel Development and his Linux System Programming. I think they would clear some mist our of my head in this area.

                1. 23

                  This blog is so consistently good and gives such concrete suggestions. Thanks.

                  1. 5

                    I’ll be swapping on summer wheels, cleaning, clay barring, and ceramic coating my car in preparation for the automotive enthusiast season. Probably going to the local car meetup on Sunday (depending on weather) and then chilling at home or with my girlfriend.

                    1. 2

                      Cool. I love reading these kinds of updates which aren’t tech-related

                    1. 1

                      I think SQL injection a solve problem with prepared statement or store procedure?

                      I have been using Golang and pretty much hand write my SQL query and prepared statement. In some place I even adopted Store Procedure. It feels nice because you write SQL once and any service can call it without worrying about how that particular service deal with SQL injection.

                      1. 1

                        For prepared statement, yes usually. Sometimes the thing you want to parameterise is not supported. For example table name is usually not supported.

                        For stored procs: not unless you use a prepared statement or escape correctly. Always possible to convince your code to pass “^C ROLLBACK; – do something” or similar as an arg to a stored proc.

                        Personally I don’t see people using prepared statements often in real life because a) it requires a new roundtrip b) their framework or driver can do the job.

                      1. 4

                        New blog post about caching and reading more about electronics (I’m at a fairly basic level still).

                        I’ve been reading Getting Started in Electronics which is really good and helping me a lot.

                        1. 2

                          A system’s purpose is what it does.

                          If a standard is so complex and intertwined that only 2-3 mega corps can implement it … then that is not an accident, it is the point.

                          I’m not sure what new thing will be born after the Web has fully become Google-tech, but considering that Mozilla is killing vertical tabs this year, I can’t wait for it to happen rather sooner than later.

                          1. 3

                            I’m not sure that was intentional with SVG. It pre-dates the HTML5 Google-driven push to roll back the separation of content and presentation and make it impossible to remove ads. SVG came to live at the same time as the XHTML push started to gain momentum. At the time, W3C was pushing to make everything XML with graceful fallback. If everything is XML, then you can embed everything in everything else. You Atom feed can contain XHTML, which can contain SVG. Because everything is XML and properly namespaced, if your feed reader doesn’t know about SVG and only understands a subset of HTML, it can render the text and the markup that it wants to and ignore everything else trivially.

                            The CSS and JavaScript parts come from this deep integration concept. SVG could define its own way of encoding text, or it could just define a text element that includes anything that XHTML supports. It could define its own way of representing styles, or it could just define names of elements and allow you to specify the strokes, fills, and so on with CSS. It could define its own animation scripting mechanism, or it could just expose a DOM and let you animate with JavaScript. Remember, at the time, CSS 2.0 was very new and CSS 1 was pretty trivial to implement. XHTML was working to reduce the number of elements and just have semantic markup, with the styling all moved to CSS, so this let you have a single parser for CSS (which was trivial to write) and have it work for styling anything.

                            My biggest problem with SVG is that it only half jumped on the XML band wagon. A lot of SVG is just PostScript in XML attributes. You can’t manipulate those structures uniformly (which is the main selling point of XML) but if they’d made each of the PostScript commands a separate XML node then they’d have easily doubled and probably quadrupled the file size. It was really an indication that XML needed a denser serialisation but none of the binary XML standards ever took off so SVG was forced to pay for the disadvantages of XML without being able to benefit from the advantages.

                            1. 1

                              I’m fighting tooth and nail against Google taking over the Internet. Hopefully I can get sizable enough to chip away at their monopoly.

                              1. 1

                                Killing vertical tabs? As in the tab tree extension? Can you link something about that please?

                              1. 2

                                I don’t see real discussion of how much it matters… It should be possible to estimate how much it matters by formulating a hypothesis and then testing it.

                                Here’s an example hypothesis: Excessive load time causes people to abort loading/interacting with the page.

                                If this is correct and some, but only some, of the load times for that site are excessive, then two things should vary with geography:

                                • the share of browsers that load images (and other supplemental page resources)
                                • the share of browsers that follow links, as indicated (imperfectly) by Referrer

                                Graphing these against estimated RTT per user should give a reasonable estimate of how strongly load time affect success (assuming success means what the hypothesis implies).

                                1. 2

                                  I suppose I’m trying to view how much it matters as the kind of opportunity cost -

                                  If I moved it to somewhere in New Jersey, and spent more, users would definitely save time in aggregate: half of roundtrips would be completed in 75ms rather than 105ms, a saving of 30%. Over several roundtrips that would probably mount up to around a sixth of a second off the average of first-time page loads, which is not too bad.

                                  Your idea is really good but I think I would struggle to find too many people who got frustrated with my sites loading time and closed the tab. It just isn’t complicated enough! I think you’d need a site that takes 10 seconds or so (as Medium does for me sometimes…)

                                  1. 1

                                    I quite agree that you’d struggle to find people who are dissatisfied with a 0.1s RTT, and I’d go further and say that answers the question of how much it matters.

                                    1. 1

                                      That would be relevant if anything could be achieved in a single roundtrip. Sadly, nothing much can be:

                                      It’s a bit worse than just [the time taken for one roundtrip]. Depending on what a user is doing they may end up making a number of those roundtrips. To download a web page usually requires five full roundtrips: one to resolve the domain name via DNS, one to establish the TCP connection, two more to set up an encrypted session with TLS and one, finally, for the page you wanted in the first place.

                                      It’s hard to imagine a more basic site than mine, for which total difference (I reckon) is about 0.2s. For other sites, with meaningful request chaining or lots of CORS preflighting to do, that value will increase. And this is on top of your request processing time which all comes out of your notional user experience “budget” for page load time (commonly agreed to be, what? About 1s for the user to feel that it’s instant?)

                                      1. 1

                                        I’ve heard about systems that do many requests and therefore end up delaying n×RTT. (I remember a support request long ago from someone who wrote “a tight loop around” an RPC.) But you’re the first person I’ve encountered who seems to think the problem is the RTT and ignore n.

                                        BTW, regarding “nothing can be done” to get below “usually five full roundtrips”. I tried now with a very distant web site that uses TLS 1.3 and several-day DNS TTLs, and saw around two roundtrips (compared to ICMP).

                                        1. 1

                                          But you’re the first person I’ve encountered who seems to think the problem is the RTT and ignore n.

                                          This is not the case and I think I’m out because for some reason you seem to be deliberately misinterpreting my comments.

                                          1. 1

                                            Sorry aboyt that. I did wonder (hence the several-day delay).

                                            FWIW I formed that impression because your posting focused entirely on the RTT and your using phrasing like “nothing much can be” about the number of round trips.

                                1. 25

                                  I think everyone who interacts with frontend javascript feels the same way. I’m afraid that in the backend he is going to see a similar kind of rapidly growing complexity too - except instead of npm hell it is k8s hell.

                                  I wish I had a cogent view on the forces that are making software engineering go this horrible way but I have no such insight. The best I can come up with is that Alan Kay quote about how professions that grow faster then their education end up susceptible to fads which, christ preserve me, I cannot even find by searching. I really hope that quote is true because it at least suggests that once the profession stops growing the symptoms might improve.

                                  1. 17

                                    I think it happens because the only way to make a simple solution is if you have a deep understanding of the problem you are trying to solve. It requires a much greater understanding than what is needed to solve it via sheer force of effort.

                                    In the majority of developers’ work they don’t have enough time to gain deep understanding of the problems they are trying to solve. The moment they have a solution, even a half-baked one, they move onto solving the next problem in an ever growing queue of work.

                                    Developers may also become bored before building up enough context to break through the complexity barrier. It can take a long time and many iterations to simplify some problems. Many developers (or their managers) lack the patience to keep working on “solved” problems after they have something that meets their needs well enough.

                                    As an industry we also have a problem with knowledge transfer. Even if a developer reaches a new level of understanding they may not be able to pass all of this onto the next generation of devs. The new devs go through the process of reinvention and relearning the same concepts, then the cycle continues.

                                    1. 12

                                      I think it happens because the only way to make a simple solution is if you have a deep understanding of the problem you are trying to solve. It requires a much greater understanding than what is needed to solve it via sheer force of effort.

                                      I agree, The best thing to look for in any professional, doctor, lawyer, coder, etc is their ability to not engage with a problem, instead solving it in a simple and non-intrusive way. The worst behavior from professionals are the folks who are going to do a lot of work no matter what. These guys look busy, and they’re deep in a bunch of technically wonky stuff that nobody understands, so naturally they look like they know what they’re doing and are doing a good job. The guy who shows up in flip-flops and after a five-minute conversation solves your problem? He’s just some smart eleck showman, probably a con man.

                                      It’s a severe problem. It’s eating our industry alive.

                                    2. 5

                                      I do have a (self-consistent, if not necessarily correct or happy) set of thoughts which explain the dynamic sufficiently for me.

                                      1. As developer productivity improves, the set of problems profitably solved by software grows faster than productivity does, so there’s demand for more developers the more productive they are.
                                      2. Software development frequently generates profits far in excess of what is needed to sustain the operation
                                      3. Organisations which enjoy profits far in excess of their operating needs become dominated by empire-building because there is no counter-pressure.
                                      4. As an empire-building manager, I need to invent plausible ways to occupy the developers I hire.
                                      5. A consultancy will recommend technologies suitable to the size of the team I have (that is, something that will require all of my staff to maintain).
                                      6. A consultancy will generally not recommend something that works with no configuration or setup required, since then they can’t sell you configuration or setup work.
                                      1. 1

                                        k8s hell

                                        Thats DevOps, not Backend? More like composer, pip, gem, all of who are better in one way or another against trashy npm and alike.

                                      1. 15

                                        There’s no such thing as a free lunch!

                                        Anyway, what’s the purpose of Cloudflare anyway? Rent a server in a good datacenter and pay for a DDoS-plan if you’re so inclined. Too many websites use Cloudflare and give it too much power over what content can be seen on the internet. Using Tor? Blocked. Coming from an IP we don’t like? Blocked. Javascript disabled? Sorry, but you really need to fill out this Captcha.

                                        On top of that, it’s one giant MITM and I am seriously shocked this hasn’t been discussed much more intensely. It would be trivial (if it hasn’t happened already or was the whole purpose of this shebang) for a five-eye-agency to wiretap it.

                                        The NSA et. al. don’t like that more and more traffic is being encrypted. It woule be a great tactic of them to spread mindshare about Cloudflare about it being almost essential and at least “good to have” for every pet-project. “Everybody loves free DDoS-protection, and Google has it too!”

                                        1. 19

                                          Anyway, what’s the purpose of Cloudflare anyway?

                                          The purpose is that they’re a CDN

                                          Rent a server in a good datacenter and pay for a DDoS-plan if you’re so inclined.

                                          This doesn’t replicate a CDN

                                          On top of that, it’s one giant MITM and I am seriously shocked this hasn’t been discussed much more intensely. It would be trivial (if it hasn’t happened already or was the whole purpose of this shebang) for a five-eye-agency to wiretap it.

                                          I don’t know about you, but the threat model for my personal website (or indeed a professional website) does not include defending against the intelligence services of my own government (“Five Eyes”). That is a nihilistic security scenario and not one I can really take seriously.

                                          For my money, I think the author of TFA has (wildly) unrealistic expectations of a free service. I’m only sorry that Cloudflare have to put up with free tier customers loudly complaining that they had a problem and needed to make at least a notional contribution in order to get it resolved.

                                          1. 9

                                            Sure, it doesn’t have to fit your threat model but by using Cloudflare you’re actively enabling the centralization of the web.

                                            1. 10

                                              In my defense I must say that I am merely passively enabling The Centralisation of The Web, at most, as I have formed no opinion of it and am taking no special action either to accelerate it or reverse it, whatever it is.

                                              1. 3

                                                What’s a good, existing, decentralized solution to DDoS protection?

                                                1. 1

                                                  Not necessary good, but very much existing and decentralized, is IPFS. Comprises quite a bit more of the stack than your standard CDN; nevertheless, it has many of the same benefits, at least as far as I understand it. There’s even a sort of IPFS dashboard (it’s FOSS!) that abstracts over most of the lower-level steps in the process.

                                                  If you are at all dismayed that the current answer to your question is “nothing”, then IPFS is definitely one project to keep an eye on.

                                                  1. 1

                                                    Ironically, one of the first results when googling about how to set up IPFS is hosted on… Cloudflare:

                                                    https://developers.cloudflare.com/distributed-web/ipfs-gateway

                                            2. 18

                                              Cloudflare’s S1 filing explains how it makes money from free users. Traffic from free users gives Cloudflare scale needed to negotiate better peering deals, and more cached sites save ISPs more money (ISPs prefer to get these free sites from a local Cloudflare pop, instead of across the world from aws-us-east-1).

                                              1. 7

                                                I’m digging for the blog post that references this, but Cloudflare in a past RCA has said that their free tier is, essentially, the canary for their deployments: changes land there first because it is better to break someone who isn’t paying for your service than someone who is.

                                                (FWIW, I don’t think this is a bad thing; I’m more than happy to let some of my sites be someone else’s guinea pig in exchange for the value Cloudflare adds.)

                                                E: Found it!

                                                https://blog.cloudflare.com/details-of-the-cloudflare-outage-on-july-2-2019/

                                                If the DOG test passes successfully code goes to PIG (as in “Guinea Pig”). This is a Cloudflare PoP where a small subset of customer traffic from non-paying customers passes through the new code.

                                                1. 4

                                                  Yes, free users sometimes get releases earlier. However, the PIG set is not all free customers, but only a small fraction. In this case “non-paying” meant “owes money”.

                                              2. 3

                                                Have to agree. Besides, their preloading page in front of websites is really annoying and I wouldn’t use that for the sake of UX. Each time I get one, I just bounce instead of waiting 5 secs.

                                              1. 9

                                                I don’t disagree with this article as it is, but I think it’s too focused on private sector applications in Silicon Valley. I have seen some great UX work done in the public sector here in the UK. “New-style” government websites are generally pretty good, even when the policy they are implementing is not.

                                                1. 3

                                                  One of his examples is the vaccine website for the public health office of New York, and I know many other vaccine websites in California, run by public health agencies, are also byzantine labyrinths. Even the website to book a test in my own town is a UX nightmare. His entire point was that the Silicon Valley is leading us down a dark path when it comes to UX.

                                                  1. 4

                                                    Sorry, yes, you’re right. I read the article somewhere else a day ago and saw it here and then commented without re-reading it. I should’ve!

                                                1. 5

                                                  These benchmarks looked suspiciously as though they didn’t even enough workers for the sync frameworks - the graph of cpu usage from gnome monitor never has more than a couple of cpus at 100% usage which is deeply suspicious.

                                                  Looking through his code on github I was not able to find anywhere that he was increasing the worker count above 1 for either sync or async so consequently he has the numbers all wrong.

                                                  If you compare 1 async worker to 1 sync worker you are implicitly allowing the async worker unlimited concurrency until it saturates a cpu but allowing the async worker no concurrency as it blocks for IO.

                                                  You might find that async performs better for certain tasks but if you find that async trounces sync Python on an embarassingly parallel task (serving web requests) then you need to look at whether you’ve made a mistake in your measurements - as I think this person has. I spent an enormous amount of time on my own benchmarks on this subject last year and discussed the problem with worker counts and concurrency there.

                                                  1. 1

                                                    That’s a compelling article you’ve written, thanks. I wonder if part of the appeal of async is that you don’t need to tune your number of processes as much. I imagine that different loads would cause larger shifts in optimum number of processes in sync frameworks compared to async. But maybe that doesn’t matter? How much worse do the sync frameworks get when you provide too many processes?

                                                    1. 1

                                                      Glad you liked it!

                                                      I think that the fact that you don’t need to tune worker counts under async is probably the key reason why they come out better in most benchmarks. In practice though, I think they are much much more fragile. I’ve seen firsthand where the production situation is like that in “Rachel By the Bay”’s article - very fragile services and weird, empirically-discovered deployment rules like “don’t let cpu go over 35% or you’ll get random timeouts”.

                                                      How much worse do the sync frameworks get when you provide too many processes?

                                                      Each process has a memory cost of your entire application image (your code+libraries+several mb for interpreter) because sadly in Python that cannot be shared. So increased memory would be a cost of having too many processes. However request processing time doesn’t really suffer from too many workers as far as I’ve seen, even when you have 5 or 6 time those required.

                                                      1. 1

                                                        Rachel’s article is great, too. Very clear about the probable causes of the issues.

                                                        Yeah, I guess I would expect sync processes not to affect processing time if they don’t wake up until the server has something for them (so long as the server has sufficient memory for working processes).

                                                        I’m writing a web server at the moment in a language that can use OS threads (Julia), so I don’t imagine that much of this will apply (I think you get all the benefits of the sync setup if you just start a new thread for every request), but it’s still interesting.

                                                  1. 7

                                                    I always wonder how much all the privacy changes going into Firefox effect measured market share. Also adblock usage, which I’d (blindly) assume to be higher on Firefox than Chrome.

                                                    1. 13

                                                      Mozilla has been placing ads in the German subway. (I’ve seen it in first in Hamburg, but I’ve also seen it in Cologne, Berlin and Munich) It says in German “This ad has no clue about who you are and where you’re coming from. Online-trackers do. Block them! And protect your privacy. With Firefox.” (Not my tweet, but searching for “firefox werbung u-bahn” yielded this tweet)

                                                      I feel that Mozilla is going all in on privacy. (Context: Germany is a very private society culturally and also due to its past. Also one of the country with the highest usage of firefox.)

                                                      1. 4

                                                        Firefox isn’t a particularly aggressive browser on privacy though, Safari and Brave are much further ahead on this and have been for a long time. I think at this point Mozilla’s claims to the contrary are false advertising - possibly literally given that they apparently have a physical marketing campaign running in Germany. Even the big feature Mozilla are trumping in this release has already been implemented by Chrome!

                                                        While I think privacy is a big motivator for lots of people and could be a big selling point of Firefox, I think consumers correctly see that Mozilla is not especially strong on privacy. Anyway I don’t see this realistically arresting the collapse in Firefox’s market share which is reduced by something like 10% in the last six months alone (ie: from 4.26% to 3.77%). On Mozilla’s current course they will probably fall to sub-1% market share in the next couple of years.

                                                        1. 10

                                                          You can dismiss this comment as biased, but I want to share my perspective as someone with a keen interest in strict privacy protections who also talks to the relevant developers first-hand. (I work on Security at Mozilla, not Privacy).

                                                          Firefox has had privacy protections like Tracking Protection, Enhanced Tracking Protection and First Party Isolation for a very, very long time. If you want aggressive privacy, you will always have to seek it for yourself. It’s seldomly in the defaults. And regardless of how effective that is, Mozilla wants to serve all users. Not just techies.

                                                          To serve all users, there’s a balance to strike with site breakage. Studies have shown that the more websites break, the less likely it is that users are going to accept the protection as a useful mechanism. In the worst case, the user will switch to a different browser that “just works”, but we’ve essentially done them a disservice. By being super strict, a vast amount of users might actually get less privacy.

                                                          So, the hard part is not being super strict on privacy (which Brave can easily do, with their techie user base), but making sure it works for your userbase. Mozilla has been able to learn from Safari’s “Intelligent Tracking Protection”, but it’s not been a pure silver bullet ready for reuse either. Safari also doesn’t have to cave in when there’s a risk of market share loss, given that they control the browser market share on iOS so tightly (aside: every browser on iOS has to use a WebKit webview. Bringing your own rendering engine is disallowed. Chrome for iOS and Firefox for iOS are using Webkit webviews)

                                                          The road to a successful implementation required many iterations, easy “report failure” buttons and lots of baking time with technical users in Firefox Beta to support major site breakage and produce meaningful bug reports.

                                                          1. 5

                                                            collapse in Firefox’s market share which is reduced by something like 10% in the last six months alone (ie: from 4.26% to 3.77%)

                                                            On desktop it’s actually increased: from 7.7% last year to 8.4% this year. A lot of the decrease in total web users is probably attributable to the increase in mobile users.

                                                            Does this matter? I don’t know; maybe not. But things do seem a bit more complex than just a single 2-dimensional chart. Also, this is still millions of people: more than many (maybe even most) popular GitHub projects.

                                                            1. 3

                                                              That’s reassuring in a sense but also baffling for me as Firefox on mobile is really good and can block ads via extensions so I really feel like if life was fair it would have a huge market share.

                                                              1. 5

                                                                And a lot of Android phones name Chrome just “Browser”; you really need to know that there’s such a thing as “Firefox” (or indeed, any other browser) in the first place. Can’t install something you don’t know exists. This is essentially the same as the whole Windows/IE thing back in the day, eventually leading to the browserchoice.eu thing.

                                                                On iOS you couldn’t even change the default browser until quite recently, and you’re still stuck with the Safari render engine of course. As far as I can tell the only reason to run Firefox on macOS is the sync with your desktop if you use Firefox.

                                                                Also, especially when looking at world-wide stats then you need to keep in mind that not everyone is from western countries. In many developing countries people are connected to the internet (usually on mobile only) and are, on average, less tech-savvy, and concepts such as privacy as we have are also a lot less well known, partly for cultural reasons, partly for educational reasons (depending a bit on the country). If you talk to a Chinese person about the Great Firewall and the like then they usually don’t really see a problem with it. It’s hard to understate how big the cultural divide can be.

                                                                Or, a slightly amusing anecdote to illustrate this: I went on a Tinder date last year (in Indonesia), and at some point she asked me what my religion was. I said that I have no religion. She just started laughing like I said something incredibly funny. Then she then asked which God I believe in. “Well, ehh, I don’t really believe in any God”. I thought she was going to choke on laughter. Just the very idea that someone doesn’t believe in God was completely alien to her; she asked me all sorts of questions about how I could possibly not have a religion 🤷 Needless to say, I don’t talk much about my religious views here (also, because blasphemy is illegal and people have been fined and even jailed over very minor remarks). Of course, this doesn’t describe all Indonesians; I also know many who hate all this religious bullshit here (those tend to be the fun ones), but it’s not the standard attitude.

                                                                So talking about privacy on the internet and “software freedom as in free speech” is probably not too effective in places where you don’t have privacy and free speech in the first place, and where these values don’t really exist in the public consciousness, which is the majority of the world (in varying degrees).

                                                                1. 3

                                                                  And a lot of Android phones name Chrome just “Browser”; you really need to know that there’s such a thing as “Firefox” (or indeed, any other browser) in the first place. Can’t install something you don’t know exists. This is essentially the same as the whole Windows/IE thing back in the day, eventually leading to the browserchoice.eu thing.

                                                                  Yes. And the good thing is: the EU commission is at it again. Google has been fined in 2018. Actually, new Android devices should now ask the user about the browser.

                                                                2. 2

                                                                  The self-destructing cookies plugin is the thing that keeps me on FireFox on Android. It’s the first sane cookie policy I’ve ever seen: When you leave a page, cookies are moved aside. Next time you visit it, all of the cookies are gone. If you lost some state that you care about (e.g. persistent login), there’s an undo button to bring them back and you can bring them back and add the site to a list that’s allowed to leave persistent cookies at the same time. I wish all browsers would make this the default policy out of the box.

                                                            2. 4

                                                              But WhatsApp is still the main way to communicate.

                                                              1. 2

                                                                That’s probably true in every country. The Germans I know are all big on signal.

                                                          1. 4

                                                            Note that Chrome implemented this earlier than Firefox: https://developers.google.com/web/updates/2020/10/http-cache-partitioning.

                                                            1. 12

                                                              It reads to me as if Chrome only partitions the http cache. Firefox does it for all network state.

                                                              1. 10

                                                                Sure, meaning only that Google no longer needs this ability and is shutting the door behind them. When Google leaves the marketplace, it will be harder for someone else to replace their advertising dominance.

                                                                1. 3

                                                                  A darkly cynical view (which I hope is not true!) is that Google no longer needing this misfeature is what made it politically possible for Firefox to implement their change.

                                                              1. 4

                                                                I’m determined to get one of the 3 draft blog posts I have 90% written out the door!

                                                                1. 8

                                                                  All but one of these function to make reviews stricter - ie: reject more stuff - but none of them really are ways to make reviews more effective. Effective does not always mean “stricter”.

                                                                  My personal experience is that there is a strong bias among programmers to feeling good that they rejected stuff and “were a tough reviewer!” and never considering the false positive side of that where you end up asking people to make a bunch of time-consuming changes based on perceived strictness but which are not really worthwhile. Talking about preventing “sub-optimally factored code” sounds like micromanagement at the review stage and does start to ring alarm bells for me!

                                                                  This one in particular really rubs me the wrong way:

                                                                  To review a change, it is important to agree on the problem to solve. Ask the author to supply a problem statement if it is not presented in the change. Only once you understand the problem statement you can evaluate the quality of a solution. The solution could be over- or under-engineered, implemented in the wrong place, or you could even disagree with the problem statement altogether.

                                                                  If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that. You need to immediately introspect about why your team process has spent time producing code that was fundamentally wrong from the beginning.

                                                                  1. 6

                                                                    If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that.

                                                                    I think there’s some nuance here in terms of where the change came from. In open source, it’s fairly common for someone to show up and drop a PR on your project with little or no explanation. That doesn’t make it bad, and it doesn’t make the person who submitted it bad. However, understanding the problem they are trying to solve is still important and may just need to happen in the code review. On the other hand, if we’re talking about a corporate setting, or a PR submitted by a regular contributor to a particular open source project, then it should already have been discussed (and if it wasn’t, there are probably larger communication issues).

                                                                    1. 2

                                                                      From personal experience, I’ve helped someone maintain a friendly fork of an open source project because they wanted a change that I definitely did not want upstream, but which solved a real problem for them (working around a bug in something else that was specific to their deployment). I think ‘this is a great implementation of what it does, but we don’t want it’ is a fine outcome for open source code review.

                                                                    2. 4

                                                                      I’ve definitely met some people who went too far (in my opinion) when it comes to requesting changes in code review. I prefer to conduct code review in person or in a freeform chat, rather than gerrit / PR conversations / etc. because I think there’s a bit of nuance to it. In my opinion things should go something like the following:

                                                                      Major bugs / functional issues: no real question here, send it back to be fixed.

                                                                      Unhandled edge case: fix if possible, but an assert guarding against the unhandled condition, and a TODO comment explaining why you might actually need it at some point, are an acceptable alternative if there’s more important work to be done.

                                                                      Violation of something explicitly set out in the company style guide: fix unless emergency.

                                                                      Refactoring opportunity / optimization opportunity / “I wouldn’t do it that way”: discuss, don’t insist. Sometimes the reviewee will agree that you have the better idea and they’ll go and make the change. Sometimes they’ll prove you wrong because they learned something during implementation that you overlooked from your lofty position. Sometimes they’ll agree that you made a good suggestion, but you’ll agree together that now isn’t the time to squeeze in the change (but the reviewee learned something, and you can come back to it later). And sometimes you’ll just agree to disagree on a matter of style. I think a lot of senior devs push too hard for having things done exactly their way, and end up wasting time and goodwill on things that are ultimately immaterial.

                                                                      1. 3

                                                                        I have almost word-for-word the exact same outlook. One trick I use (which it sounds like you already know) is to just phrase pretty much every piece of feedback as a question rather than a demand. Most people are good-natured and will either make the change you’re implying or will explain why not (often convincingly). One of the very worst things you can do is get up on your high horse about something in the code before hearing about it from the author.

                                                                        1. 1

                                                                          Yeah, I tend towards that as well. I don’t hold fast to it, but I agree. Even in the most straightforward cases, “hey, won’t the code blow up here if this variable is less than zero?” sets a much better tone than “this is broken.” Not to mention, it makes me look better in case I made a mistake and there isn’t actually a bug.

                                                                      2. 3

                                                                        On the contrary, I have a really hard time rejecting stuff. I suspect a lot of people in the industry are probably similar to me (shy, imposter syndrome, etc). I’m sure there are just as many on the other side of the spectrum (socially tone-deaf, cocksure, etc).

                                                                        It has to be pretty “bad” for me to reject it or ask for a large refactoring of their approach, for better or worse. Most of the time, if I don’t see actually errors/bugs with the approach, I try my best not to shit on the whole approach just because it was not the way I like it done.

                                                                        1. 1

                                                                          On the contrary, I have a really hard time rejecting stuff. I suspect a lot of people in the industry are probably similar to me (shy, imposter syndrome, etc).

                                                                          I have reviewed and subject my code from/to hundreds of other developers and what I observed is that you suspicion off. By and large, the majority, think 90%+, get high on power and are eager to reject patches for everything and nothing. White space, bracket style, indentation, style, naming conventions, linting and other secondary details always end up taking up like 75% of the work involved in code reviews. Grand parent is spot on.

                                                                          To make the assertion that everyone is in position to review everyones code is a very silly premise, and can only result into non sense like this. We then get sprinkled with posts like this one on “how to do it right”, as if they have the magical formula and everyone else is doing it wrong.

                                                                          In the occasions I had ownership of the code I end up asking fellow developers to not submit the code to review unless they feel uncertain about it or need additional input for any specific reason.

                                                                          “At least two pairs of eyes”, “a last line of safety before the main trunk” and other silly mantras are non sense to account for lack of competence or code quality. You can throw 20 pairs of eyes at it, if those 20 are bad developers, than it is worst than just one pair of eyes. There is no shortcut to competence.

                                                                          Furthermore, the idea that the whole code can be reviewed bit by bit doesn’t make sense. You can have 20 patches that all look perfectly fine by themselves, but when you put them together are a disaster.

                                                                          An experiment for anyone to try: gather the reject ration of your coworkers and compare with how you would rate them in terms of skill/competence. These are usually inversely proportional.

                                                                          1. 1

                                                                            White space, bracket style, indentation, style, naming conventions, linting and other secondary details always end up taking up like 75% of the work involved in code reviews.

                                                                            Other than naming conventions, humans shouldn’t be wasting their time reviewing any of those things anyway; automated tools should be rejecting those things before a code review request even goes out.

                                                                            On teams I’ve been on where we’ve had automated checks for machine-detectable violations of the project’s style conventions, code reviews have tended to stay much more focused on the stuff that matters.

                                                                        2. 1

                                                                          If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that. You need to immediately introspect about why your team process has spent time producing code that was fundamentally wrong from the beginning.

                                                                          I’d agree strongly with this. Having implementation approach be part of the PR feedback cycle makes development inordinately time consuming and adversarial; technical approach needs to be either (ideally) ingrained into the day-to-day communication on your team through some mechanism like slack, daily sync meetings, etc. or else formally deliberated on before the time is spent to build a complete solution. It’s better to take the time to have a half an hour meeting than it is letting a two-day change be implemented in an unacceptable fashion.

                                                                          1. 2

                                                                            It’s better to take the time to have a half an hour meeting than it is letting a two-day change be implemented in an unacceptable fashion.

                                                                            Doesn’t that depend on how often the problem happens? It is, I think, not better to have a few dozen half-hour meetings over the course of a year when you would only have prevented one or two rejected PRs. Those meetings don’t come at zero cost to the team.

                                                                            1. 1

                                                                              It does depend on how often the problem happens but I think the probabilities you’re implicitly assuming aren’t quite right (IME, ofc). Disagreements over approach at review stage are quite common if the person was not involved at the design stage, largely people there are just a lot of ways to do something. In the worst cases you will find a senior or lead developer will be too busy to attend design sessions but will sit as gatekeeper on PRs. That tends to lead to lots of rework and a very slow effective pace of improvement.

                                                                              To make it explicit with some numbers: getting a design agreed usually only takes a 10-15 minute discussion between a 2-3 people, one of which can do it and one of which can review. If you skip that it I would say that you have a ~15% chance - and that’s conservative - of drama raised at review stage and rework. Solving for x, if your piece of work takes >5 hours, you should always have the design session. Obviously this is approximate maths, I just hope to give a sense of when the conversation is worthwhile and when it’s not.

                                                                        1. 4

                                                                          This is the perfect length and a good laugh.

                                                                          1. 6

                                                                            Firstly I’m looking for work again after 6 months of being a full time dad for my son. I have an interview later today so here’s hoping!

                                                                            Second I am (er…rather gingerly) showing people my side project and getting feedback. In short it’s an interface on top of your browser bookmarks which shows you discussions about your bookmarks, links between them, full text search, and so on. It’s sync-based so you can bookmark stuff on your phone and it will pick them up too - so long as you are also using browser sync. If anyone here has a few minutes to check it out, I’d love to hear any thoughts. Here’s a sharelink to one of my bookmarks to give a taste: https://quarchive.com/shares/idD41uGRH0RIMTPYBgM9PhXg

                                                                            1. 1

                                                                              Oh, the discussions feature looks interesting! Though, as if it wasn’t enough of a sinkhole & addiction browsing lobste.rs & HN, at least for me…… :/

                                                                              1. 1

                                                                                At least with this you’re looking at discussions of things you’re interested in I suppose! (I spend a fair amount of time browsing through comments on things I know nothing about)

                                                                              2. 1

                                                                                Good luck on your interview!

                                                                                1. 1

                                                                                  Thanks mate! That one didn’t work out but very close on others!