1. 1

    I think SQL injection a solve problem with prepared statement or store procedure?

    I have been using Golang and pretty much hand write my SQL query and prepared statement. In some place I even adopted Store Procedure. It feels nice because you write SQL once and any service can call it without worrying about how that particular service deal with SQL injection.

    1. 1

      For prepared statement, yes usually. Sometimes the thing you want to parameterise is not supported. For example table name is usually not supported.

      For stored procs: not unless you use a prepared statement or escape correctly. Always possible to convince your code to pass “^C ROLLBACK; – do something” or similar as an arg to a stored proc.

      Personally I don’t see people using prepared statements often in real life because a) it requires a new roundtrip b) their framework or driver can do the job.

    1. 4

      New blog post about caching and reading more about electronics (I’m at a fairly basic level still).

      I’ve been reading Getting Started in Electronics which is really good and helping me a lot.

      1. 2

        A system’s purpose is what it does.

        If a standard is so complex and intertwined that only 2-3 mega corps can implement it … then that is not an accident, it is the point.

        I’m not sure what new thing will be born after the Web has fully become Google-tech, but considering that Mozilla is killing vertical tabs this year, I can’t wait for it to happen rather sooner than later.

        1. 2

          I’m not sure that was intentional with SVG. It pre-dates the HTML5 Google-driven push to roll back the separation of content and presentation and make it impossible to remove ads. SVG came to live at the same time as the XHTML push started to gain momentum. At the time, W3C was pushing to make everything XML with graceful fallback. If everything is XML, then you can embed everything in everything else. You Atom feed can contain XHTML, which can contain SVG. Because everything is XML and properly namespaced, if your feed reader doesn’t know about SVG and only understands a subset of HTML, it can render the text and the markup that it wants to and ignore everything else trivially.

          The CSS and JavaScript parts come from this deep integration concept. SVG could define its own way of encoding text, or it could just define a text element that includes anything that XHTML supports. It could define its own way of representing styles, or it could just define names of elements and allow you to specify the strokes, fills, and so on with CSS. It could define its own animation scripting mechanism, or it could just expose a DOM and let you animate with JavaScript. Remember, at the time, CSS 2.0 was very new and CSS 1 was pretty trivial to implement. XHTML was working to reduce the number of elements and just have semantic markup, with the styling all moved to CSS, so this let you have a single parser for CSS (which was trivial to write) and have it work for styling anything.

          My biggest problem with SVG is that it only half jumped on the XML band wagon. A lot of SVG is just PostScript in XML attributes. You can’t manipulate those structures uniformly (which is the main selling point of XML) but if they’d made each of the PostScript commands a separate XML node then they’d have easily doubled and probably quadrupled the file size. It was really an indication that XML needed a denser serialisation but none of the binary XML standards ever took off so SVG was forced to pay for the disadvantages of XML without being able to benefit from the advantages.

          1. 1

            Killing vertical tabs? As in the tab tree extension? Can you link something about that please?

            1. 1

              I’m fighting tooth and nail against Google taking over the Internet. Hopefully I can get sizable enough to chip away at their monopoly.

            1. 2

              I don’t see real discussion of how much it matters… It should be possible to estimate how much it matters by formulating a hypothesis and then testing it.

              Here’s an example hypothesis: Excessive load time causes people to abort loading/interacting with the page.

              If this is correct and some, but only some, of the load times for that site are excessive, then two things should vary with geography:

              • the share of browsers that load images (and other supplemental page resources)
              • the share of browsers that follow links, as indicated (imperfectly) by Referrer

              Graphing these against estimated RTT per user should give a reasonable estimate of how strongly load time affect success (assuming success means what the hypothesis implies).

              1. 2

                I suppose I’m trying to view how much it matters as the kind of opportunity cost -

                If I moved it to somewhere in New Jersey, and spent more, users would definitely save time in aggregate: half of roundtrips would be completed in 75ms rather than 105ms, a saving of 30%. Over several roundtrips that would probably mount up to around a sixth of a second off the average of first-time page loads, which is not too bad.

                Your idea is really good but I think I would struggle to find too many people who got frustrated with my sites loading time and closed the tab. It just isn’t complicated enough! I think you’d need a site that takes 10 seconds or so (as Medium does for me sometimes…)

                1. 1

                  I quite agree that you’d struggle to find people who are dissatisfied with a 0.1s RTT, and I’d go further and say that answers the question of how much it matters.

                  1. 1

                    That would be relevant if anything could be achieved in a single roundtrip. Sadly, nothing much can be:

                    It’s a bit worse than just [the time taken for one roundtrip]. Depending on what a user is doing they may end up making a number of those roundtrips. To download a web page usually requires five full roundtrips: one to resolve the domain name via DNS, one to establish the TCP connection, two more to set up an encrypted session with TLS and one, finally, for the page you wanted in the first place.

                    It’s hard to imagine a more basic site than mine, for which total difference (I reckon) is about 0.2s. For other sites, with meaningful request chaining or lots of CORS preflighting to do, that value will increase. And this is on top of your request processing time which all comes out of your notional user experience “budget” for page load time (commonly agreed to be, what? About 1s for the user to feel that it’s instant?)

                    1. 1

                      I’ve heard about systems that do many requests and therefore end up delaying n×RTT. (I remember a support request long ago from someone who wrote “a tight loop around” an RPC.) But you’re the first person I’ve encountered who seems to think the problem is the RTT and ignore n.

                      BTW, regarding “nothing can be done” to get below “usually five full roundtrips”. I tried now with a very distant web site that uses TLS 1.3 and several-day DNS TTLs, and saw around two roundtrips (compared to ICMP).

                      1. 1

                        But you’re the first person I’ve encountered who seems to think the problem is the RTT and ignore n.

                        This is not the case and I think I’m out because for some reason you seem to be deliberately misinterpreting my comments.

                        1. 1

                          Sorry aboyt that. I did wonder (hence the several-day delay).

                          FWIW I formed that impression because your posting focused entirely on the RTT and your using phrasing like “nothing much can be” about the number of round trips.

              1. 25

                I think everyone who interacts with frontend javascript feels the same way. I’m afraid that in the backend he is going to see a similar kind of rapidly growing complexity too - except instead of npm hell it is k8s hell.

                I wish I had a cogent view on the forces that are making software engineering go this horrible way but I have no such insight. The best I can come up with is that Alan Kay quote about how professions that grow faster then their education end up susceptible to fads which, christ preserve me, I cannot even find by searching. I really hope that quote is true because it at least suggests that once the profession stops growing the symptoms might improve.

                1. 17

                  I think it happens because the only way to make a simple solution is if you have a deep understanding of the problem you are trying to solve. It requires a much greater understanding than what is needed to solve it via sheer force of effort.

                  In the majority of developers’ work they don’t have enough time to gain deep understanding of the problems they are trying to solve. The moment they have a solution, even a half-baked one, they move onto solving the next problem in an ever growing queue of work.

                  Developers may also become bored before building up enough context to break through the complexity barrier. It can take a long time and many iterations to simplify some problems. Many developers (or their managers) lack the patience to keep working on “solved” problems after they have something that meets their needs well enough.

                  As an industry we also have a problem with knowledge transfer. Even if a developer reaches a new level of understanding they may not be able to pass all of this onto the next generation of devs. The new devs go through the process of reinvention and relearning the same concepts, then the cycle continues.

                  1. 12

                    I think it happens because the only way to make a simple solution is if you have a deep understanding of the problem you are trying to solve. It requires a much greater understanding than what is needed to solve it via sheer force of effort.

                    I agree, The best thing to look for in any professional, doctor, lawyer, coder, etc is their ability to not engage with a problem, instead solving it in a simple and non-intrusive way. The worst behavior from professionals are the folks who are going to do a lot of work no matter what. These guys look busy, and they’re deep in a bunch of technically wonky stuff that nobody understands, so naturally they look like they know what they’re doing and are doing a good job. The guy who shows up in flip-flops and after a five-minute conversation solves your problem? He’s just some smart eleck showman, probably a con man.

                    It’s a severe problem. It’s eating our industry alive.

                  2. 5

                    I do have a (self-consistent, if not necessarily correct or happy) set of thoughts which explain the dynamic sufficiently for me.

                    1. As developer productivity improves, the set of problems profitably solved by software grows faster than productivity does, so there’s demand for more developers the more productive they are.
                    2. Software development frequently generates profits far in excess of what is needed to sustain the operation
                    3. Organisations which enjoy profits far in excess of their operating needs become dominated by empire-building because there is no counter-pressure.
                    4. As an empire-building manager, I need to invent plausible ways to occupy the developers I hire.
                    5. A consultancy will recommend technologies suitable to the size of the team I have (that is, something that will require all of my staff to maintain).
                    6. A consultancy will generally not recommend something that works with no configuration or setup required, since then they can’t sell you configuration or setup work.
                    1. 1

                      k8s hell

                      Thats DevOps, not Backend? More like composer, pip, gem, all of who are better in one way or another against trashy npm and alike.

                    1. 15

                      There’s no such thing as a free lunch!

                      Anyway, what’s the purpose of Cloudflare anyway? Rent a server in a good datacenter and pay for a DDoS-plan if you’re so inclined. Too many websites use Cloudflare and give it too much power over what content can be seen on the internet. Using Tor? Blocked. Coming from an IP we don’t like? Blocked. Javascript disabled? Sorry, but you really need to fill out this Captcha.

                      On top of that, it’s one giant MITM and I am seriously shocked this hasn’t been discussed much more intensely. It would be trivial (if it hasn’t happened already or was the whole purpose of this shebang) for a five-eye-agency to wiretap it.

                      The NSA et. al. don’t like that more and more traffic is being encrypted. It woule be a great tactic of them to spread mindshare about Cloudflare about it being almost essential and at least “good to have” for every pet-project. “Everybody loves free DDoS-protection, and Google has it too!”

                      1. 19

                        Anyway, what’s the purpose of Cloudflare anyway?

                        The purpose is that they’re a CDN

                        Rent a server in a good datacenter and pay for a DDoS-plan if you’re so inclined.

                        This doesn’t replicate a CDN

                        On top of that, it’s one giant MITM and I am seriously shocked this hasn’t been discussed much more intensely. It would be trivial (if it hasn’t happened already or was the whole purpose of this shebang) for a five-eye-agency to wiretap it.

                        I don’t know about you, but the threat model for my personal website (or indeed a professional website) does not include defending against the intelligence services of my own government (“Five Eyes”). That is a nihilistic security scenario and not one I can really take seriously.

                        For my money, I think the author of TFA has (wildly) unrealistic expectations of a free service. I’m only sorry that Cloudflare have to put up with free tier customers loudly complaining that they had a problem and needed to make at least a notional contribution in order to get it resolved.

                        1. 9

                          Sure, it doesn’t have to fit your threat model but by using Cloudflare you’re actively enabling the centralization of the web.

                          1. 10

                            In my defense I must say that I am merely passively enabling The Centralisation of The Web, at most, as I have formed no opinion of it and am taking no special action either to accelerate it or reverse it, whatever it is.

                            1. 3

                              What’s a good, existing, decentralized solution to DDoS protection?

                              1. 1

                                Not necessary good, but very much existing and decentralized, is IPFS. Comprises quite a bit more of the stack than your standard CDN; nevertheless, it has many of the same benefits, at least as far as I understand it. There’s even a sort of IPFS dashboard (it’s FOSS!) that abstracts over most of the lower-level steps in the process.

                                If you are at all dismayed that the current answer to your question is “nothing”, then IPFS is definitely one project to keep an eye on.

                                1. 1

                                  Ironically, one of the first results when googling about how to set up IPFS is hosted on… Cloudflare:

                                  https://developers.cloudflare.com/distributed-web/ipfs-gateway

                          2. 18

                            Cloudflare’s S1 filing explains how it makes money from free users. Traffic from free users gives Cloudflare scale needed to negotiate better peering deals, and more cached sites save ISPs more money (ISPs prefer to get these free sites from a local Cloudflare pop, instead of across the world from aws-us-east-1).

                            1. 7

                              I’m digging for the blog post that references this, but Cloudflare in a past RCA has said that their free tier is, essentially, the canary for their deployments: changes land there first because it is better to break someone who isn’t paying for your service than someone who is.

                              (FWIW, I don’t think this is a bad thing; I’m more than happy to let some of my sites be someone else’s guinea pig in exchange for the value Cloudflare adds.)

                              E: Found it!

                              https://blog.cloudflare.com/details-of-the-cloudflare-outage-on-july-2-2019/

                              If the DOG test passes successfully code goes to PIG (as in “Guinea Pig”). This is a Cloudflare PoP where a small subset of customer traffic from non-paying customers passes through the new code.

                              1. 4

                                Yes, free users sometimes get releases earlier. However, the PIG set is not all free customers, but only a small fraction. In this case “non-paying” meant “owes money”.

                            2. 3

                              Have to agree. Besides, their preloading page in front of websites is really annoying and I wouldn’t use that for the sake of UX. Each time I get one, I just bounce instead of waiting 5 secs.

                            1. 9

                              I don’t disagree with this article as it is, but I think it’s too focused on private sector applications in Silicon Valley. I have seen some great UX work done in the public sector here in the UK. “New-style” government websites are generally pretty good, even when the policy they are implementing is not.

                              1. 3

                                One of his examples is the vaccine website for the public health office of New York, and I know many other vaccine websites in California, run by public health agencies, are also byzantine labyrinths. Even the website to book a test in my own town is a UX nightmare. His entire point was that the Silicon Valley is leading us down a dark path when it comes to UX.

                                1. 4

                                  Sorry, yes, you’re right. I read the article somewhere else a day ago and saw it here and then commented without re-reading it. I should’ve!

                              1. 5

                                These benchmarks looked suspiciously as though they didn’t even enough workers for the sync frameworks - the graph of cpu usage from gnome monitor never has more than a couple of cpus at 100% usage which is deeply suspicious.

                                Looking through his code on github I was not able to find anywhere that he was increasing the worker count above 1 for either sync or async so consequently he has the numbers all wrong.

                                If you compare 1 async worker to 1 sync worker you are implicitly allowing the async worker unlimited concurrency until it saturates a cpu but allowing the async worker no concurrency as it blocks for IO.

                                You might find that async performs better for certain tasks but if you find that async trounces sync Python on an embarassingly parallel task (serving web requests) then you need to look at whether you’ve made a mistake in your measurements - as I think this person has. I spent an enormous amount of time on my own benchmarks on this subject last year and discussed the problem with worker counts and concurrency there.

                                1. 1

                                  That’s a compelling article you’ve written, thanks. I wonder if part of the appeal of async is that you don’t need to tune your number of processes as much. I imagine that different loads would cause larger shifts in optimum number of processes in sync frameworks compared to async. But maybe that doesn’t matter? How much worse do the sync frameworks get when you provide too many processes?

                                  1. 1

                                    Glad you liked it!

                                    I think that the fact that you don’t need to tune worker counts under async is probably the key reason why they come out better in most benchmarks. In practice though, I think they are much much more fragile. I’ve seen firsthand where the production situation is like that in “Rachel By the Bay”’s article - very fragile services and weird, empirically-discovered deployment rules like “don’t let cpu go over 35% or you’ll get random timeouts”.

                                    How much worse do the sync frameworks get when you provide too many processes?

                                    Each process has a memory cost of your entire application image (your code+libraries+several mb for interpreter) because sadly in Python that cannot be shared. So increased memory would be a cost of having too many processes. However request processing time doesn’t really suffer from too many workers as far as I’ve seen, even when you have 5 or 6 time those required.

                                    1. 1

                                      Rachel’s article is great, too. Very clear about the probable causes of the issues.

                                      Yeah, I guess I would expect sync processes not to affect processing time if they don’t wake up until the server has something for them (so long as the server has sufficient memory for working processes).

                                      I’m writing a web server at the moment in a language that can use OS threads (Julia), so I don’t imagine that much of this will apply (I think you get all the benefits of the sync setup if you just start a new thread for every request), but it’s still interesting.

                                1. 7

                                  I always wonder how much all the privacy changes going into Firefox effect measured market share. Also adblock usage, which I’d (blindly) assume to be higher on Firefox than Chrome.

                                  1. 13

                                    Mozilla has been placing ads in the German subway. (I’ve seen it in first in Hamburg, but I’ve also seen it in Cologne, Berlin and Munich) It says in German “This ad has no clue about who you are and where you’re coming from. Online-trackers do. Block them! And protect your privacy. With Firefox.” (Not my tweet, but searching for “firefox werbung u-bahn” yielded this tweet)

                                    I feel that Mozilla is going all in on privacy. (Context: Germany is a very private society culturally and also due to its past. Also one of the country with the highest usage of firefox.)

                                    1. 4

                                      But WhatsApp is still the main way to communicate.

                                      1. 2

                                        That’s probably true in every country. The Germans I know are all big on signal.

                                      2. 4

                                        Firefox isn’t a particularly aggressive browser on privacy though, Safari and Brave are much further ahead on this and have been for a long time. I think at this point Mozilla’s claims to the contrary are false advertising - possibly literally given that they apparently have a physical marketing campaign running in Germany. Even the big feature Mozilla are trumping in this release has already been implemented by Chrome!

                                        While I think privacy is a big motivator for lots of people and could be a big selling point of Firefox, I think consumers correctly see that Mozilla is not especially strong on privacy. Anyway I don’t see this realistically arresting the collapse in Firefox’s market share which is reduced by something like 10% in the last six months alone (ie: from 4.26% to 3.77%). On Mozilla’s current course they will probably fall to sub-1% market share in the next couple of years.

                                        1. 10

                                          You can dismiss this comment as biased, but I want to share my perspective as someone with a keen interest in strict privacy protections who also talks to the relevant developers first-hand. (I work on Security at Mozilla, not Privacy).

                                          Firefox has had privacy protections like Tracking Protection, Enhanced Tracking Protection and First Party Isolation for a very, very long time. If you want aggressive privacy, you will always have to seek it for yourself. It’s seldomly in the defaults. And regardless of how effective that is, Mozilla wants to serve all users. Not just techies.

                                          To serve all users, there’s a balance to strike with site breakage. Studies have shown that the more websites break, the less likely it is that users are going to accept the protection as a useful mechanism. In the worst case, the user will switch to a different browser that “just works”, but we’ve essentially done them a disservice. By being super strict, a vast amount of users might actually get less privacy.

                                          So, the hard part is not being super strict on privacy (which Brave can easily do, with their techie user base), but making sure it works for your userbase. Mozilla has been able to learn from Safari’s “Intelligent Tracking Protection”, but it’s not been a pure silver bullet ready for reuse either. Safari also doesn’t have to cave in when there’s a risk of market share loss, given that they control the browser market share on iOS so tightly (aside: every browser on iOS has to use a WebKit webview. Bringing your own rendering engine is disallowed. Chrome for iOS and Firefox for iOS are using Webkit webviews)

                                          The road to a successful implementation required many iterations, easy “report failure” buttons and lots of baking time with technical users in Firefox Beta to support major site breakage and produce meaningful bug reports.

                                          1. 5

                                            collapse in Firefox’s market share which is reduced by something like 10% in the last six months alone (ie: from 4.26% to 3.77%)

                                            On desktop it’s actually increased: from 7.7% last year to 8.4% this year. A lot of the decrease in total web users is probably attributable to the increase in mobile users.

                                            Does this matter? I don’t know; maybe not. But things do seem a bit more complex than just a single 2-dimensional chart. Also, this is still millions of people: more than many (maybe even most) popular GitHub projects.

                                            1. 3

                                              That’s reassuring in a sense but also baffling for me as Firefox on mobile is really good and can block ads via extensions so I really feel like if life was fair it would have a huge market share.

                                              1. 5

                                                And a lot of Android phones name Chrome just “Browser”; you really need to know that there’s such a thing as “Firefox” (or indeed, any other browser) in the first place. Can’t install something you don’t know exists. This is essentially the same as the whole Windows/IE thing back in the day, eventually leading to the browserchoice.eu thing.

                                                On iOS you couldn’t even change the default browser until quite recently, and you’re still stuck with the Safari render engine of course. As far as I can tell the only reason to run Firefox on macOS is the sync with your desktop if you use Firefox.

                                                Also, especially when looking at world-wide stats then you need to keep in mind that not everyone is from western countries. In many developing countries people are connected to the internet (usually on mobile only) and are, on average, less tech-savvy, and concepts such as privacy as we have are also a lot less well known, partly for cultural reasons, partly for educational reasons (depending a bit on the country). If you talk to a Chinese person about the Great Firewall and the like then they usually don’t really see a problem with it. It’s hard to understate how big the cultural divide can be.

                                                Or, a slightly amusing anecdote to illustrate this: I went on a Tinder date last year (in Indonesia), and at some point she asked me what my religion was. I said that I have no religion. She just started laughing like I said something incredibly funny. Then she then asked which God I believe in. “Well, ehh, I don’t really believe in any God”. I thought she was going to choke on laughter. Just the very idea that someone doesn’t believe in God was completely alien to her; she asked me all sorts of questions about how I could possibly not have a religion 🤷 Needless to say, I don’t talk much about my religious views here (also, because blasphemy is illegal and people have been fined and even jailed over very minor remarks). Of course, this doesn’t describe all Indonesians; I also know many who hate all this religious bullshit here (those tend to be the fun ones), but it’s not the standard attitude.

                                                So talking about privacy on the internet and “software freedom as in free speech” is probably not too effective in places where you don’t have privacy and free speech in the first place, and where these values don’t really exist in the public consciousness, which is the majority of the world (in varying degrees).

                                                1. 3

                                                  And a lot of Android phones name Chrome just “Browser”; you really need to know that there’s such a thing as “Firefox” (or indeed, any other browser) in the first place. Can’t install something you don’t know exists. This is essentially the same as the whole Windows/IE thing back in the day, eventually leading to the browserchoice.eu thing.

                                                  Yes. And the good thing is: the EU commission is at it again. Google has been fined in 2018. Actually, new Android devices should now ask the user about the browser.

                                                2. 2

                                                  The self-destructing cookies plugin is the thing that keeps me on FireFox on Android. It’s the first sane cookie policy I’ve ever seen: When you leave a page, cookies are moved aside. Next time you visit it, all of the cookies are gone. If you lost some state that you care about (e.g. persistent login), there’s an undo button to bring them back and you can bring them back and add the site to a list that’s allowed to leave persistent cookies at the same time. I wish all browsers would make this the default policy out of the box.

                                        1. 4

                                          Note that Chrome implemented this earlier than Firefox: https://developers.google.com/web/updates/2020/10/http-cache-partitioning.

                                          1. 12

                                            It reads to me as if Chrome only partitions the http cache. Firefox does it for all network state.

                                            1. 10

                                              Sure, meaning only that Google no longer needs this ability and is shutting the door behind them. When Google leaves the marketplace, it will be harder for someone else to replace their advertising dominance.

                                              1. 3

                                                A darkly cynical view (which I hope is not true!) is that Google no longer needing this misfeature is what made it politically possible for Firefox to implement their change.

                                            1. 4

                                              I’m determined to get one of the 3 draft blog posts I have 90% written out the door!

                                              1. 8

                                                All but one of these function to make reviews stricter - ie: reject more stuff - but none of them really are ways to make reviews more effective. Effective does not always mean “stricter”.

                                                My personal experience is that there is a strong bias among programmers to feeling good that they rejected stuff and “were a tough reviewer!” and never considering the false positive side of that where you end up asking people to make a bunch of time-consuming changes based on perceived strictness but which are not really worthwhile. Talking about preventing “sub-optimally factored code” sounds like micromanagement at the review stage and does start to ring alarm bells for me!

                                                This one in particular really rubs me the wrong way:

                                                To review a change, it is important to agree on the problem to solve. Ask the author to supply a problem statement if it is not presented in the change. Only once you understand the problem statement you can evaluate the quality of a solution. The solution could be over- or under-engineered, implemented in the wrong place, or you could even disagree with the problem statement altogether.

                                                If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that. You need to immediately introspect about why your team process has spent time producing code that was fundamentally wrong from the beginning.

                                                1. 6

                                                  If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that.

                                                  I think there’s some nuance here in terms of where the change came from. In open source, it’s fairly common for someone to show up and drop a PR on your project with little or no explanation. That doesn’t make it bad, and it doesn’t make the person who submitted it bad. However, understanding the problem they are trying to solve is still important and may just need to happen in the code review. On the other hand, if we’re talking about a corporate setting, or a PR submitted by a regular contributor to a particular open source project, then it should already have been discussed (and if it wasn’t, there are probably larger communication issues).

                                                  1. 2

                                                    From personal experience, I’ve helped someone maintain a friendly fork of an open source project because they wanted a change that I definitely did not want upstream, but which solved a real problem for them (working around a bug in something else that was specific to their deployment). I think ‘this is a great implementation of what it does, but we don’t want it’ is a fine outcome for open source code review.

                                                  2. 4

                                                    I’ve definitely met some people who went too far (in my opinion) when it comes to requesting changes in code review. I prefer to conduct code review in person or in a freeform chat, rather than gerrit / PR conversations / etc. because I think there’s a bit of nuance to it. In my opinion things should go something like the following:

                                                    Major bugs / functional issues: no real question here, send it back to be fixed.

                                                    Unhandled edge case: fix if possible, but an assert guarding against the unhandled condition, and a TODO comment explaining why you might actually need it at some point, are an acceptable alternative if there’s more important work to be done.

                                                    Violation of something explicitly set out in the company style guide: fix unless emergency.

                                                    Refactoring opportunity / optimization opportunity / “I wouldn’t do it that way”: discuss, don’t insist. Sometimes the reviewee will agree that you have the better idea and they’ll go and make the change. Sometimes they’ll prove you wrong because they learned something during implementation that you overlooked from your lofty position. Sometimes they’ll agree that you made a good suggestion, but you’ll agree together that now isn’t the time to squeeze in the change (but the reviewee learned something, and you can come back to it later). And sometimes you’ll just agree to disagree on a matter of style. I think a lot of senior devs push too hard for having things done exactly their way, and end up wasting time and goodwill on things that are ultimately immaterial.

                                                    1. 3

                                                      I have almost word-for-word the exact same outlook. One trick I use (which it sounds like you already know) is to just phrase pretty much every piece of feedback as a question rather than a demand. Most people are good-natured and will either make the change you’re implying or will explain why not (often convincingly). One of the very worst things you can do is get up on your high horse about something in the code before hearing about it from the author.

                                                      1. 1

                                                        Yeah, I tend towards that as well. I don’t hold fast to it, but I agree. Even in the most straightforward cases, “hey, won’t the code blow up here if this variable is less than zero?” sets a much better tone than “this is broken.” Not to mention, it makes me look better in case I made a mistake and there isn’t actually a bug.

                                                    2. 3

                                                      On the contrary, I have a really hard time rejecting stuff. I suspect a lot of people in the industry are probably similar to me (shy, imposter syndrome, etc). I’m sure there are just as many on the other side of the spectrum (socially tone-deaf, cocksure, etc).

                                                      It has to be pretty “bad” for me to reject it or ask for a large refactoring of their approach, for better or worse. Most of the time, if I don’t see actually errors/bugs with the approach, I try my best not to shit on the whole approach just because it was not the way I like it done.

                                                      1. 1

                                                        On the contrary, I have a really hard time rejecting stuff. I suspect a lot of people in the industry are probably similar to me (shy, imposter syndrome, etc).

                                                        I have reviewed and subject my code from/to hundreds of other developers and what I observed is that you suspicion off. By and large, the majority, think 90%+, get high on power and are eager to reject patches for everything and nothing. White space, bracket style, indentation, style, naming conventions, linting and other secondary details always end up taking up like 75% of the work involved in code reviews. Grand parent is spot on.

                                                        To make the assertion that everyone is in position to review everyones code is a very silly premise, and can only result into non sense like this. We then get sprinkled with posts like this one on “how to do it right”, as if they have the magical formula and everyone else is doing it wrong.

                                                        In the occasions I had ownership of the code I end up asking fellow developers to not submit the code to review unless they feel uncertain about it or need additional input for any specific reason.

                                                        “At least two pairs of eyes”, “a last line of safety before the main trunk” and other silly mantras are non sense to account for lack of competence or code quality. You can throw 20 pairs of eyes at it, if those 20 are bad developers, than it is worst than just one pair of eyes. There is no shortcut to competence.

                                                        Furthermore, the idea that the whole code can be reviewed bit by bit doesn’t make sense. You can have 20 patches that all look perfectly fine by themselves, but when you put them together are a disaster.

                                                        An experiment for anyone to try: gather the reject ration of your coworkers and compare with how you would rate them in terms of skill/competence. These are usually inversely proportional.

                                                        1. 1

                                                          White space, bracket style, indentation, style, naming conventions, linting and other secondary details always end up taking up like 75% of the work involved in code reviews.

                                                          Other than naming conventions, humans shouldn’t be wasting their time reviewing any of those things anyway; automated tools should be rejecting those things before a code review request even goes out.

                                                          On teams I’ve been on where we’ve had automated checks for machine-detectable violations of the project’s style conventions, code reviews have tended to stay much more focused on the stuff that matters.

                                                      2. 1

                                                        If you are ever in the situation where someone has spent time implementing something and their entire approach gets rejected at the final review stage you should absolutely not be patting yourself on the back for that. You need to immediately introspect about why your team process has spent time producing code that was fundamentally wrong from the beginning.

                                                        I’d agree strongly with this. Having implementation approach be part of the PR feedback cycle makes development inordinately time consuming and adversarial; technical approach needs to be either (ideally) ingrained into the day-to-day communication on your team through some mechanism like slack, daily sync meetings, etc. or else formally deliberated on before the time is spent to build a complete solution. It’s better to take the time to have a half an hour meeting than it is letting a two-day change be implemented in an unacceptable fashion.

                                                        1. 2

                                                          It’s better to take the time to have a half an hour meeting than it is letting a two-day change be implemented in an unacceptable fashion.

                                                          Doesn’t that depend on how often the problem happens? It is, I think, not better to have a few dozen half-hour meetings over the course of a year when you would only have prevented one or two rejected PRs. Those meetings don’t come at zero cost to the team.

                                                          1. 1

                                                            It does depend on how often the problem happens but I think the probabilities you’re implicitly assuming aren’t quite right (IME, ofc). Disagreements over approach at review stage are quite common if the person was not involved at the design stage, largely people there are just a lot of ways to do something. In the worst cases you will find a senior or lead developer will be too busy to attend design sessions but will sit as gatekeeper on PRs. That tends to lead to lots of rework and a very slow effective pace of improvement.

                                                            To make it explicit with some numbers: getting a design agreed usually only takes a 10-15 minute discussion between a 2-3 people, one of which can do it and one of which can review. If you skip that it I would say that you have a ~15% chance - and that’s conservative - of drama raised at review stage and rework. Solving for x, if your piece of work takes >5 hours, you should always have the design session. Obviously this is approximate maths, I just hope to give a sense of when the conversation is worthwhile and when it’s not.

                                                      1. 4

                                                        This is the perfect length and a good laugh.

                                                        1. 6

                                                          Firstly I’m looking for work again after 6 months of being a full time dad for my son. I have an interview later today so here’s hoping!

                                                          Second I am (er…rather gingerly) showing people my side project and getting feedback. In short it’s an interface on top of your browser bookmarks which shows you discussions about your bookmarks, links between them, full text search, and so on. It’s sync-based so you can bookmark stuff on your phone and it will pick them up too - so long as you are also using browser sync. If anyone here has a few minutes to check it out, I’d love to hear any thoughts. Here’s a sharelink to one of my bookmarks to give a taste: https://quarchive.com/shares/idD41uGRH0RIMTPYBgM9PhXg

                                                          1. 1

                                                            Oh, the discussions feature looks interesting! Though, as if it wasn’t enough of a sinkhole & addiction browsing lobste.rs & HN, at least for me…… :/

                                                            1. 1

                                                              At least with this you’re looking at discussions of things you’re interested in I suppose! (I spend a fair amount of time browsing through comments on things I know nothing about)

                                                            2. 1

                                                              Good luck on your interview!

                                                              1. 1

                                                                Thanks mate! That one didn’t work out but very close on others!

                                                            1. 3

                                                              Yes for my own site the RSS feed is the most downloaded thing by far. I don’t include the page content so bandwidth is not a problem but it is weird that many readers are clearly ignoring the TTL and just requesting it over and over.

                                                              1. 4

                                                                Misconfigured readers/central aggregators have been a bane of RSS almost since the beginning. Developers don’t always read specs, or test.

                                                                1. 9

                                                                  Even worse, commonly used readers like ttrss adamantly refuse to do the bare minimum like using etags. ttrss makes requests every 15 minutes to a feed that only really updates once per day at most. I’d have to convince all of my readers that use ttrss to fix their settings instead of them being a good net citizen and following recommendations to make the protocol work better.

                                                                  1. 6

                                                                    That’s horrific. I would even say antisocial. Shit like this degrades the entire ecosystem — RSS is already kludgy enough as it is without developers petulantly refusing to adopt the few scalability measures available.

                                                                    Plus, the developer’s response is childish:

                                                                    … get a better hosting provider or something. your sperging out over literal bytes wasted on http redirects on your blog and stuff is incredibly pathetic.

                                                                    Pathetic, indeed.

                                                                      1. 2

                                                                        I kind of liked ttrss until I understood how the developer acts in general. I’ve moved to miniflux since.

                                                                        I’ve also understood that ttrss had some issues regarding security that the developer just refused to address or fix, due to reasons.

                                                                      2. 1

                                                                        haha! tiny tiny rss happens to be the #1 useragent for my site!

                                                                        1. 1

                                                                          Having HTTP features (cache-control and last-modified) duplicated in the RSS spec is really annoying. Developers don’t want to write a custom cache for RSS feeds. I don’t know why supporting redundant caching measures encoded in XML would make a piece of software better. Why wouldn’t a HTTP cache be sufficient?

                                                                          1. 3

                                                                            AFAIK we are talking about HTTP caching in this thread. There are sites that don’t include headers like Etag or even Last-Modified, and there are clients that ignore them and just send unconditional requests every time.

                                                                            There are one or two RSS properties related to this, but the only one I remember specifies a suggested poll interval. That does overlap to some degree with Cache-Control, but I don’t remember what the reasoning was for it. I’m certainly not going to defend RSS as a format, it’s pretty crude and the “official” specs are awful.

                                                                          2. 1

                                                                            Tempting to vary on a response header so one could prepend a bad netizen warning post to the top of the list for those readers that are being problematic.

                                                                      1. 4

                                                                        Learning Erlang by reading https://learnyousomeerlang.com/

                                                                        1. 1

                                                                          If that book leaves you looking for more, Programming Erlang is a classic: https://www.goodreads.com/book/show/808814.Programming_Erlang

                                                                          1. 1

                                                                            I am reading this too. I have slogged through the functional programming stuff and am about up to where the concurrency material starts. Hopefully it’s about to become a lot more interesting!

                                                                            1. 0

                                                                              Enjoy! Erlang is such a fun language. If you have Ruby experience I can recommend Elixir as well, they’re really closely related.

                                                                            1. 9

                                                                              I really think ‘package manager for ${LANGUAGE}’ is one of the most insidious anti-features that’s eating software engineering at the moment. First, it assumes that all of your dependencies are written in the same language. Modern software is written in multiple languages (Chome and Firefox were both at around 30 languages last count), how well do all of these compose? Integrating them with the build system makes things worse. CMake’ support for Objective-C is still very new and has bugs (hopefully they’ll reopen some of the ones that I filed that were closed because Objective-C was not supported) and Objective-C can be built with exactly the same compiler and compiler driver and linked with the same linker as C and C++. For anything else, it is even worse.

                                                                              Even if you do have all of your dependencies in the same language (for example, if you’re a Go developer and don’t want to suffer through the nightmare that is Go’s C interop layer), now you are causing pain for packagers and folks doing security audits. Consider a simple example: How would the FreeBSD package infrastructure respond to a bug in OpenSSL? I presume most Linux distros have similar machinery but this is the one I’m most familiar with. A quick look at the package graph finds everything that (indirectly) requires the OpenSSL package as a build or run-time dependency. This gives a safe over-approximation of everything that needs fixing. The VuXML entry is published with the vulnerability and so pkg audit can automatically notify any of the users of affected packages. When there’s a fix, the OpenSSL package is updated and everything that depends on it is rebuilt (typically, in fact, everything is rebuilt because it only takes about 24 hours on a big beefy machine and it’s easier than figuring out what needs incremental building).

                                                                              Now what happens if your program has used some language-specific package manager to include OpenSSL? If you’ve done dynamic linking, you may pick up the packaged version but if you’ve statically linked then you won’t and so the package maintainer needs to separately update your program. Not too bad for one program, but imagine if 10,000 packages (around a third of the total) did it.

                                                                              Now spare a thought for whoever is responsible for the security process for a large deployment. After Heartbleed, everyone on *NIX systems could check if they were vulnerable by querying their package manager and if they had a package set from after the fix was pushed out they were fine. With things pulling in dependencies that the package manager has no visibility into, this is much worse.

                                                                              Now imagine that you want your language-specific package manager to integrate with the platform-specific package manager. Now you have an N:M problem, where every language builds a package manager and needs an interop layer for every target OS.

                                                                              What I really want is a portable format for expressing dependencies (canonical name, version, any build options that I need enabled, and so on) and querying any package manager to provide them. On Windows or macOS, this could just grab NuGet packages or CocoaPods that I bundle with my program, on platforms with a uniform package management story it could simply drive that.

                                                                              1. 2

                                                                                That’s why Bazel and Nix is very important, both are language agnostic package managers. I use Bazel in a setup where you have C dependencies, Python dependencies and Swift dependencies, and it works like a charm.

                                                                                That has been said, these language agnostic package managers have to integrate with the rest of other language specific package managers to really take off unfortunately.

                                                                                1. 1

                                                                                  Now what happens if your program has used some language-specific package manager to include OpenSSL? If you’ve done dynamic linking, you may pick up the packaged version but if you’ve statically linked then you won’t and so the package maintainer needs to separately update your program. Not too bad for one program, but imagine if 10,000 packages (around a third of the total) did it.

                                                                                  I think the answer to this is: if you’ve done something over and above what your operating system’s package manager provides, you’re responsible for it and your operating system is not.

                                                                                  Language specific package managers were around and in active use at the time of Heartbleed and didn’t pose any special problems. I worked on a big public website at the time and we updated the operating system packages, then the relevant Ruby packages and then regenerated certs. It was an operational nightmare for many reasons but I have to say that, from memory, finding and updating the relevant software two times was not one of them.

                                                                                  The problem seems to revolve around static linking (and containerisation too - which has the same properties as static linking for this discussion) and not language-specific package managers.

                                                                                1. 14

                                                                                  I hope that 2021 will be a year of slower technology popularisation/adoption as the lag between the technology that is available and the most developer’s understanding is unsustainably large.

                                                                                  Please, nothing new! We don’t need it, we’re all still getting to grips with the old stuff!

                                                                                  1. 5

                                                                                    At its launch, the content is stored in HTML format. This is OK — we all know a little HTML — but it is not the most convenient format to edit and write, especially if you are creating a sizable new page from scratch. Most people find Markdown easier to write than HTML, so we want to eventually move to storing our core content in Markdown (or maybe some other format) rather than HTML.

                                                                                    I suspect they will find this hard. Markdown works great when your needs are simple, like entering a comment on a website. I wouldn’t like to prepare a big set of interlinked documents with it. Depending on which specific tool you use you’ll run into problems like:

                                                                                    • no support for front matter (title, author, other metadata). If it does have support for front matter, good luck getting that into the title, meta description, etc tags without some other kind of templating on top
                                                                                    • very limited support for facilitating styling (eg adding a class to a table, which the most popular Python markdown library can’t do)
                                                                                    • no ability to have macros/sub-fragments, for example when you want some kind of common element midway down the page.

                                                                                    There are other markup languages that can do all of this but they don’t have the popularity of markdown or the simplicity. Sphinx’s restructured text can do all of this, but I have to look up the syntax all the time and all of the cheatsheets that rank well in google have problems. Curious to see what they do.

                                                                                    1. 2

                                                                                      I think restructured text is also interesting but suffers a lot from the surrounding infrastructure.

                                                                                      Sphinx is really thankless work, and I appreciate all that’s gone into it, but from the maintainer’s mouth themselves, it’s not meant to be used as a library, just an application. And ReST without Sphinx and its extensions doesn’t go very far.

                                                                                      It would be amazing if this somehow lead to some investment into ReST being more usable as a general tool instead of, effectively, just the “sphinx markup language for python docs”

                                                                                      1. 2

                                                                                        I feel the exact other way around. I don’t really like RST and don’t know anyone who does really like it but many people use it because Sphinx is so good.