1. 6

    Either there was a shift in what’s considered “good” in Python, or it’s just my bias towards such articles, but this quote is quintessentially un-Pythonic, in my book:

    If the code is widespread enough you’re almost guaranteed that someone is using your code in a way you didn’t intend.


    In general, what’s wrong with people using your code in ways you haven’t foreseen? You are responsible for making it clear how to use your code, but if people still do it differently, may be they have their reasons? In Python it was called the “consenting adults” principle.

    In fact, if you think about it, people using your library always have more context about their use cases than you when you’re writing it!

    Another quote, “but what about beginners to Python or the library?”, shows a rather condescending assumption where a grizzled pro is supposed to write an iron-clad, bug-free library before handing it down to infantile juniors who would undoubtedly wreak all sorts of havoc, casting a shade on the perfect library code (poetic exaggeration). Sometimes it’s the other way around.


    In particular, the suggested function signature:

    def process_data(data, /, *, encoding="ascii"):
    

    … forces my to parrot the name encoding in situation like this:

    data, encoding = load_data()
    process_data(data, encoding=encoding)
    

    … only because you didn’t trust me in taking care about readability of my own code.

    1. 5

      There’s “haven’t foreseen” as in, “ah that’s interesting” but also as in “that’s totally not how I though about it so it’s not tested and will accidentally get broken in the next minor revision”. It’s the second one that a stricter approach can help you avoid.

      1.  

        How can you know the more strict API is correct, and you won’t discover a need for an incompatible change in it later? It’s juts a completely orthogonal question.

        You are not responsible for how your users use your code. The moment they started dependent on your library they become responsible for testing how it behaves in all their uses.

        1.  

          How can you know the more strict API is correct,

          It’s not really about “correct” in some absolute sense. Just “less likely to cause issues later”. You’re of course not responsible for downstream code, but at the same time you probably want to avoid unexpected issues if you can tighten up your published signatures.

          If you wouldn’t want to purposefully break downstream code, why would you avoid methods to protect from accidental breakage?

          1.  

            How do you know if your tightening up will avoid or solve problems. Generally, you can’t know the uses you’re preventing are going to be useful to the caller. Especially stylistic choices! All you can do is to clearly document what the function expects and what it returns. But don’t police your future users.

            1.  

              But don’t police your future users.

              On the flip-side, don’t try to dictate what decisions open-source maintainers make.

          2.  

            kw-only arguments provide a stricter signature, and just like type annotations, these are helpful to check for correctness, and also to evolve a codebase over time.

            as an example, imagine that the library evolves into using input_encoding= and output_encoding= arguments for this function, because of new insights:

            • with kw-only arguments, the new arguments can be added, and it’s easy to keep encoding= working for backwards compatibility. code using the library will keep working without changes.

            • without kw-only arguments, what would process(data, 'ascii', 'utf-8') mean? input and output encoding, as the new approach suggests? nope! it suddenly starts to matter which arguments are declared first, and that depends on the history of the code, which noone should care about anymore for new uses. but the legacy encoding argument has to be forever kept in the same position because otherwise it would break backward-compatibility. so process(data, 'ascii', 'utf-8') would specify both encoding and the new input_encoding argument (or output_encoding, depending on the order in which they are specified; see where this is going?). this is surprising and perhaps even wrong since encoding and input_encoding semantically overlap. sure, one could write extra checks and code to guard against this, but it’s a hassle and the end result is not nice.

            kw-only arguments are a great way to avoid footguns like this altogether.

            taking this approach has absolutely nothing to do with ‘annoying’ people who don’t like to type an argument name. code is read way more often than it’s written. ‘readability counts’ and ‘explicit is better than implicit’, as the python mantras go.

            there’s absolutely nothing wrong with ‘parrotting’ code like encoding=encoding. unlike positional and positional-or-keyword arguments, kw-only args make it clear at the call site what their meaning is, even for people who never saw the code before. that’s a good thing.

            and as a bonus it also becomes impossible to accidentally swap arguments around, something that type checkers cannot reliably check (because the arguments may be of the same type).

      1. 57

        The developer of these libraries intentionally introduced an infinite loop that bricked thousands of projects that depend on ’colors and ‘faker’.

        I wonder if the person who wrote this actually knows what “bricked” means.

        But beyond the problem of not understanding the difference between “bricked” and “broke”, this action did not break any builds that were set up responsibly; only builds which tell the system “just give me whatever version you feel like regardless of whether it works” which like … yeah, of course things are going to break if you do that! No one should be surprised.

        Edit: for those who are not native English speakers, “bricked” refers to a change (usually in firmware on an embedded device) which not only causes the device to be non-functional, but also breaks whatever update mechanisms you would use to get it back into a good state. It means the device is completely destroyed and must be replaced since it cannot be used as anything but a brick.

        GitHub has reportedly suspended the developer’s account

        Hopefully this serves as a wakeup call for people about what a tremendously bad idea it is to have all your code hosted by a single company. Better late than never.

        1. 25

          There have been plenty of wakeup calls for people using Github, and I doubt one additional one will change the minds of very many people (which doesn’t make it any less of a good idea for people to make their code hosting infrastructure independent from Github). The developer was absolutely trolling (in the best sense of the word) and a lot of people have made it cleared that they’re very eager for Github to deplatform trolls.

          I don’t blame him certainly; he’s entitled to do whatever he wants with the free software he releases, including trolling by releasing deliberately broken commits in order to express his displeasure at companies using his software without compensating him in the way he would like.

          The right solution here is for any users of these packages to do exactly what the developer suggested and fork them without the broken commits. If npm (or cargo, or any other programming language ecosystem package manager) makes it difficult for downstream clients to perform that fork, this is an argument for changing npm in order to make that easier. Build additional functionality into npm to make it easier to switch away from broken or otherwise-unwanted specific versions of a package anywhere in your project’s dependency tree, without having to coordinate this with other package maintainers.

          1. 31

            The developer was absolutely trolling (in the best sense of the word)

            To the extent there is any good trolling, it consists of saying tongue-in-cheek things to trigger people with overly rigid ideas. Breaking stuff belonging to people who trusted you is not good in any way.

            I don’t blame him certainly; he’s entitled to do whatever he wants with the free software he releases, including trolling by releasing deliberately broken commits in order

            And GitHub was free to dump his account for his egregious bad citizenship. I’m glad they did, because this kind of behavior undermines the kind of collaborative trust that makes open source work.

            to express his displeasure at companies using his software without compensating him in the way he would like.

            Take it from me: the way to get companies to compensate you “in six figures” for your code is to release your code commercially, not open source. Or to be employed by said companies. Working on free software and then whining that companies use it for free is dumbshittery of an advanced level.

            1. 32

              No I think the greater fool is the one who can’t tolerate changes like this in free software.

              1. 1

                It’s not foolish to trust, initially. What’s foolish is to keep trusting after you’ve been screwed. (That’s the lesson of the Prisoner’s Dilemma.)

                A likely lesson companies will draw from this is that free software is a risk, and that if you do use it, stick to big-name reputable projects that aren’t built on a house of cards of tiny libraries by unknown people. That’s rather bad news for ecosystems like node or RubyGems or whatever.

              2. 12

                Working on free software and then whining that companies use it for free is dumbshittery of an advanced level.

                Thankyou. This is the point everybody seems to be missing.

                1. 49

                  The author of these libraries stopped whining and took action.

                  1. 3

                    Worked out a treat, too.

                    1. 5

                      I mean, it did. Hopefully companies will start moving to software stacks where people are paid for their effort and time.

                      1. 6

                        He also set fire to the building making bombs at home, maybe he’s not a great model.

                        1. 3

                          Not if you’re being responsible and pinning your deps though?

                          Even if that weren’t true though, the maintainer doesn’t have any obligation to companies using their software. If the company used the software without acquiring a support contract, then that’s just a risk of business that the company should have understood. If they didn’t, that’s their fault, not the maintainer’s - companies successfully do this kind of risk/reward calculus all the time in other areas, successfully.

                          1. 1

                            I know there are news reports of a person with the same name being taken into custody in 2020 where components that could be used for making bombs were found, but as far as I know, no property damage occurred then. Have there been later reports?

                          2. 3

                            Yeah, like proprietary or in-house software. Great result for open source.

                            Really, if I were a suit at a company and learned that my product was DoS’d by source code we got from some random QAnon nutjob – that this rando had the ability to push malware into his Git repo and we’d automatically download and run it – I’d be asking hard questions about why my company uses free code it just picked up off the sidewalk, instead of paying a summer intern a few hundred bucks to write an equivalent library to printf ANSI escape sequences or whatever.

                            That’s inflammatory language, not exactly my viewpoint but I’m channeling the kind of thing I’d expect a high-up suit to say.

                  2. 4

                    There have been plenty of wakeup calls for people using Github, and I doubt one additional one will change the minds of very many people

                    Each new incident is another feather. For some, it’s the last one to break the camel’s back.

                    1. 4

                      in order to express his displeasure at companies using his software without compensating him in the way he would like.

                      This sense of entitlement is amusing. This people totally miss the point of free software. They make something that many people find useful and use (Very much thanks to the nature of being released with a free license, mind you), then they feel in their right to some sort of material/monetary compensatiom.

                      This is not miss universe contest. It’s not too hard to understand that had this project been non free, it would have probably not gotten anywhere. This is the negative side of GitHub. GitHub has been an enormously valuable resource for free software. Unfortunately, when it grows so big, it will inevitably also attract this kind of people that only like the free aspect of free software when it benefits them directly.

                      1. 28

                        This people totally miss the point of free software.

                        An uncanny number of companies (and people employed by said companies) also totally miss the point of free software. They show up in bug trackers all entitled like the license they praise in all their “empowering the community” slides doesn’t say THE SOFTWARE IS PROVIDED “AS IS” in all fscking caps. If you made a list of all the companies to whom the description “companies that only like the free aspect of free software when it benefits them directly” doesn’t apply, you could apply a moderately efficient compression algorithm and it would fit in a boot sector.

                        I don’t want to defend what the author did – as someone else put it here, it’s dumbshittery of an advanced level. But if entitlement were to earn you an iron “I’m an asshole” pin, we’d have to mine so much iron ore on account of the software industry that we’d trigger a second Iron Age.

                        This isn’t only on the author, it’s what happens when corporate entitlement meets open source entitlement. All the entitled parties in this drama got exactly what they deserved IMHO.

                        Now, one might argue that what this person did affected not just all those entitled product managers who had some tough explaining to do to their suit-wearing bros, but also a bunch of good FOSS “citizens”, too. That’s absolutely right, but while this may have been unprofessional, the burden of embarrassment should be equally shared by the people who took a bunch of code developed by an independent, unpaid developer, in their spare time – in other words, a hobby project – without any warranty, and then baked it in their super professional codebases without any contingency plan for “what if all that stuff written in all caps happens?”. This happened to be intentional but a re-enactment of this drama is just one half-drunk evening hacking session away.

                        It’s not like they haven’t been warned – when a new dependency is proposed, that part is literally the first one that’s read, and it’s reviewed by a legal team whose payment figures are eye-watering. You can’t build a product based only on the good parts of FOSS. Exploiting FOSS software only when it benefits yourself may also be assholery of an advanced level, but hoping that playing your part shields you from all the bad parts of FOSS is naivety of an advanced level, and commercial software development tends to punish that.

                        1. 4

                          They show up in bug trackers all entitled like the license they praise in all their “empowering the community” slides doesn’t say THE SOFTWARE IS PROVIDED “AS IS” in all fscking caps

                          Slides about F/OSS don’t say that because expensive proprietary software has exactly the same disclaimer. You may have an SLA that requires bugs to be fixed within a certain timeframe, but outside of very specialised markets you’ll be very hard pressed to find any software that comes with any kind of liability for damage caused by bugs.

                          1. 1

                            Well… I meant the license, not the slides :-P. Indeed, commercial licenses say pretty much the same thing. However, at least in my experience, the presence of that disclaimer is not quite as obvious with commercial software – barring, erm, certain niches.

                            Your average commercial license doesn’t require proprietary vendors to issue refunds, provide urgent bugfixes or stick by their announced deadlines for fixes and veatures. But the practical constraints of staying in business are pretty good at compelling them to do some of these things.

                            I’ve worked both with and without SLAs so I don’t want to sing praises to commercial vendors – some of them fail miserably, and I’ve seen countless open source projects that fix security issues in less time than it takes even competent large vendors to call a meeting to decide a release schedule for the fix. But expecting the same kind of commitment and approachability from Random J. Hacker is just not a very good idea. Discounting pathological arseholes and know-it-alls, there are perfectly human and understandable reasons why the baseline of what you get is just not the same when you’re getting it from a development team with a day job, a bus factor of 1, and who may have had a bad day and has no job description that says “be nice to customers even if you had a bad day or else”.

                            The universe npm has spawned is particularly susceptible to this. It’s a universe where adding a PNG to JPG conversion function pulls fourty dependencies, two of which are different and slightly incompatible libraries which handle emojis just in case someone decided to be cute with file names, and they’re going to get pulled even if the first thing your application does is throw non-alphanumeric characters out of any string, because they’re nth order dependencies with no config overrides. There’s a good chance that no matter what your app does, 10% of your dependencies are one-person resume-padding efforts that turned out to be unexpectedly useful and are now being half-heartedly maintained largely because you never know when you’ll have to show someone you’re a JavaScript ninja guru in this economy. These packages may well have the same “no warranty” sticker that large commercial vendors put on theirs, but the practical consequences of having that sticker on the box often differ a lot.

                            Edit: to be clear, I’m not trying to say “proprietary – good and reliable, F/OSS – slow and clunky”, we all know a lot of exceptions to both. What I meant to point out is that the typical norms of business-to-business relations just don’t uniformly apply to independent F/OSS devs, which makes the “no warranty” part of the license feel more… intense, I guess.

                        2. 12

                          The entitlement sentiment goes both ways. Companies that expect free code and get upset if the maintainer breaks backward compatibility. Since when is that an obligation to behave responsibly?

                          When open source started, there wasn’t that much money involved and things were very much in the academic spirit of sharing knowledge. That created a trove of wealth that companies are just happy to plunder now.

                        3. 1

                          releasing deliberately broken commits in order to express his displeasure at companies using his software without compensating him in the way he would like.

                          Was that honestly the intent? Because in that case: what hubris! These libraries were existing libraries translated to JS. He didn’t do any of the hard work.

                        4. 8

                          There is further variation on the “bricked” term, at least in the Android hacker’s community. You might hear things like “soft bricked” which refers to a device that has the normal installation / update method not working, but could be recovered through additional tools, or perhaps using JTAG to reprogram the bootloader.

                          There is also “hard bricked” which indicates something completely irreversible, such as changing the fuse programming so that it won’t boot from eMMC anymore. Or deleting necessary keys from the secure storage.

                          1. 3

                            this action did not break any builds that were set up responsibly; only builds which tell the system “just give me whatever version you feel like regardless of whether it works” which like … yeah, of course things are going to break if you do that! No one should be surprised.

                            OK, so, what’s a build set up responsibly?

                            I’m not sure what the expectations are for packages on NPM, but the changes in that colors library were published with an increment only to the patch version. When trusting the developers (and if you don’t, why would you use their library?), not setting in stone the patch version in your dependencies doesn’t seem like a bad idea.

                            1. 26

                              When trusting the developers (and if you don’t, why would you use their library?), not setting in stone the patch version in your dependencies doesn’t seem like a bad idea.

                              No, it is a bad idea. Even if the developer isn’t actively malicious, they might’ve broken something in a minor update. You shouldn’t ever blindly update a dependency without testing afterwards.

                              1. 26

                                Commit package-lock.json like all of the documentation tells you to, and don’t auto-update dependencies without running CI.

                                1. 3

                                  And use npm shrinkwrap if you’re distributing apps and not libraries, so the lockfile makes it into the registry package.

                                2. 18

                                  Do you really think that a random developer, however well intentioned, is really capable of evaluating whether or not any given change they make will have any behavior-observable impact on downstream projects they’re not even aware of, let alone have seen the source for and have any idea how it consumes their project?

                                  I catch observable breakage coming from “patch” revisions easily a half dozen times a year or more. All of it accidental “oh we didn’t think about that use-case, we don’t consume it like that” type stuff. It’s truly impossible to avoid for anything but the absolute tiniest of API surface areas.

                                  The only sane thing to do is to use whatever your tooling’s equivalent of a lock file is to strictly maintain the precise versions used for production deploys, and only commit changes to that lock file after a full re-run of the test suite against the new library version, patch or not (and running your eyeballs over a diff against the previous version of its code would be wise, as well).

                                  It’s wild to me that anyone would just let their CI slip version updates into a deploy willynilly.

                                  1. 11

                                    This neatly shows why Semver is a broken religion: you can’t just rely on a version number to consider changes to be non-broken. A new version is a new version and must be tested without any assumptions.

                                    To clarify, I’m not against specifying dependencies to automatically update to new versions per se, as long as there’s a CI step to build and test the whole thing before it goes it production, to give you a chance to pin the broken dependency to a last-known-good version.

                                    1. 7

                                      Semver doesn’t guarantee anything though and doesn’t promise anything. It’s more of an indicator of what to expect. Sure, you should test new versions without any assumptions, but that doesn’t say anything about semver. What that versioning scheme allows you to do though is put minor/revision updates straight into ci and an automatic PR, while blocking major ones until manual action.

                                    2. 6

                                      The general form of the solution is this:

                                      1. Download whatever source code you are using into a secure versioned repository that you control.

                                      2. Test every version that you consider using for function before you commit to it in production/deployment/distribution.

                                      3. Build your system from specific versions, not from ‘last update’.

                                      4. Keep up to date on change logs, security lists, bug trackers, and whatever else is relevant.

                                      5. Know what your back-out procedure is.

                                      These steps apply to all upstream sources: language modules, libraries, OS packages… dependency management is crucial.

                                      1. 3

                                        Amazon does this. Almost no-one else does this, but that’s a choice with benefits (saving the set up effort mostly) and consequences (all of this here)

                                      2. 6

                                        When trusting the developers (and if you don’t, why would you use their library?)

                                        If you trust the developers, why not give them root on your laptop? After all, you’re using their library so you must trust them, right?

                                        1. 7

                                          There’s levels to trust.

                                          I can believe you’rea good person by reading your public posts online, but I’m not letting you babysit my kids.

                                      3. 2

                                        Why wouldn’t this behavior be banned by any company?

                                        1. 2

                                          How do they ban them, they’re not paying them? Unless you mean the people who did not pin the dependencies?

                                          1. 4

                                            I think it is bannable on any platform, because it is malicious behavior - that means he intentionally caused harm to people. It’s not about an exchange of money, it’s about intentional malice.

                                          2. 1

                                            Because it’s his code and even the license says “no guarantees” ?

                                            1. 2

                                              The behavior was intentionally malicious. It’s not about violating a contract or guarantee. For example, if he just decided that he was being taken advantage of and removed the code, I don’t think that would require a ban. But he didn’t do that - he added an infinite loop to purposefully waste people’s time. That is intentional harm, that’s not just providing a library of poor quality with no guarantee.

                                              Beyond that, if that loop went unnoticed on a build server and costed the company money, I think he should be legally responsible for those damages.

                                        1. 15

                                          Note that Rust does this automatically for you since Rust 1.18, released in 2017. By coincidence, the example case used, (u8, u16, u8), is exactly same in the Rust release note.

                                          1. 2

                                            Does anyone know if Go could do this? And was it ever considered?

                                            1. 4

                                              As far as I know, there is nothing in the spec that guarantees struct order layout, and the only way to really observe it is to use the unsafe package which is not covered by the Go 1 compatibility promise. So technically it could.

                                              That said, changing it now would break too much code that uses structs with cgo/assembly/syscalls, so I doubt it will happen any time soon. If it ever does, I’d expect it will come with a similar annotation to Rust’s #[repr(C)], at the very least.

                                              Here’s an issue where these things have been considered: https://github.com/golang/go/issues/36606

                                            2. 1

                                              That seems rather brutal for binary compatibility or when interacting with C. I assume it can be turned off on a struct by struct basis?

                                              1. 12

                                                Yes, #[repr(C)], it’s right there in the linked release notes :-)

                                                1. 3

                                                  And all the details (and other representations) are documented: https://doc.rust-lang.org/reference/type-layout.html#representations

                                            1. 15

                                              Only one thing jumped out on me which I definitely didn’t agree:

                                              Rust makes it especially easy to refactor

                                              Specifically, extracting a piece of code into a separate function could actually be harder in Rust than in some other languages with type inference due to Rust’s treatment of any function as an API boundary and its insistence on spelling out all argument types explicitly. Sometimes the types of intermittent values could be rather unwieldy (cough iterator adapters cough).

                                              1. 25

                                                I think Rust is easier to refactor correctly. When refactoring something, if it compiles, it often works right away. Maybe it’s just me but when refactoring C++ or Python I always introduce little mismatches that I don’t find until runtime.

                                                1. 8

                                                  Indeed, it’s all relative. I have found having a good strong type system can embolden me to do things I wouldn’t even attempt in a dynamic language. I don’t have first hand experience doing heavy refactoring in Rust; I’ve only used it for a few small projects, but this has been my experience with Haskell, OCaml, and heck even Go, when the type system is capable of covering the invariants you care about. I do find that for a lot of my big refactors in Haskell, a decent chunk of the time it does take has been sunk into updating type annotations that are in principle optional, so I think type inference would help here – but I’ve also found that in larger codebases having the types actually written down for most top-level functions makes reading the code easier. This seems like something a good IDE could solve for you, but I’ve never really taken to IDEs, and the languages that have good inference have not historically had good IDE support anyway (though it’s been improving).

                                                  1. 2

                                                    Tests work at runtime, have better covering tests :-)

                                                    1. 6

                                                      Why would I debug minor type mismatches like a peasant when the compiler can instantaneously tell me the exact location of the issue?

                                                      Worse, when refactoring C++, changes can introduce subtle lifetime issues that even good tests may not trigger. ASAN can help, but then I’m debugging ASAN errors instead of the Rust compiler pointing me to the exact line of code causing a problem.

                                                      I write good tests, and lots of them. But running a test suite is not a substitute for type and lifetime checking in my eyes.

                                                      1. 4

                                                        Indeed, my somewhat pathological case that I’ve come to point to is SQLite; the thoroughness of their testing practices borders on the absurd (which is a good thing):

                                                        https://sqlite.org/testing.html

                                                        the SQLite library consists of approximately 143.4 KSLOC of C code. […] By comparison, the project has 640 times as much test code and test scripts.

                                                        And yet, memory safety bugs are found somwhat regularly:

                                                        https://www.sqlite.org/cves.html#cvetab

                                                        The page makes a good argument that most of these don’t really count as vulnerabilities, but they are still bugs that would be impossible to even get past the type checker in Rust, which somehow evaded what is the most rigorous testing regime I have ever heard of, on any project in any language.

                                                        To be sure, there are many kinds of bugs in many domains that type checkers aren’t great at helping with, much less ruling out entirely. But there are also many where they are a great fit, and in those cases they are both more complete than any possible test suite and also much less effort to make use of.

                                                        Neither testing nor type checkers should be the only tool in your toolkit.

                                                  2. 5

                                                    I like type inference within a function body, but global inference seems to invariably make debugging more difficult. Even if your editor shows you type annotations, these annotations are the most abstract types that the type system supports rather than the types a programmer intended. Moreover, it seems like managing compatibility would be a big pain if any change made inside of the function body would quietly change its signature (I’m sure tooling exists which could check for this, but I have a hard time imagining that tooling is more pleasant to work with than simple, by-human-for-human annotations).

                                                    1. 4

                                                      My experience is that I can feel free to start a big refactoring in Rust, even when I’m working on a new feature in parallel, because I feel secure: I won’t forget anything, the compiler is guiding me step by step and makes me converge towards the correct refactored code. It’s “fearless refactoring”.

                                                      1. 2

                                                        I agree with you, refactoring in Rust is quite painful. The author is not wrong about the compiler telling you what changes you need to make once you change a signature or a type, but as you say, I think is way harder that other languages…

                                                        1. 1

                                                          No, no, I didn’t imply it was painful in general. I just noticed this particular difficulty.

                                                      1. 4

                                                        From the ‘About’ page …

                                                        This is a fork of gddo, the software which previously powered the now-defunct godoc.org, operated by Drew DeVault. The source code is available on sr.ht.

                                                        Anyone here have context around the situation that led to the fork? I must have entirely missed this (but that’s not surprising as I only tinker with Golang).

                                                        1. 20

                                                          Godoc.org used to be an independent, open source site that Google bought just to cancel and replace with proprietary bloat that takes >10x longer to render, and won’t display documentation for except for licenses deemed acceptable by Google, in particular it rejects public domain and WTFPL. Some people dislike that and decided to do something about it.

                                                          I never cared about “silly” licenses like the WTFPL, but now I am tempted to license all my code under WTFPL. Google won’t allow the use of WTFPL code internally, which for me is absolutely great.

                                                          1. 31

                                                            Godoc.org used to be an independent, open source site that Google bought just to cancel and replace with proprietary bloat …

                                                            You posted the same lies on HN. Please stop.

                                                            Longer reply there: https://news.ycombinator.com/item?id=29314711.

                                                            1. 6

                                                              But the part about licenses is correct.

                                                              1. 3

                                                                Something like this would be better then, if one wanted to be inflammatory but more correct:

                                                                Godoc.org used to be an independent, open source site that Google agreed to take stewardship over, just to cancel and replace with proprietary bloat that takes >10x longer to render, and won’t display documentation for except for licenses deemed acceptable by Google, in particular it rejects public domain and WTFPL. Some people dislike that and decided to do something about it.

                                                                EDIT: I see from the rest of the hacker news discussion that the replacement isn’t proprietary. I can’t comment on the bloat/performance aspect, new site seems fast to me. That really only leaves the licensing stuff.

                                                                1. 6

                                                                  If people haven’t granted the right to display the documentation then sites probably shouldn’t display it, right?

                                                                  1. 1

                                                                    Still not really true. First off, a public domain dedication is not a licence. Second, CC0-1.0 is accepted.

                                                                2. 6

                                                                  WTFPL is problematic in general because the public domain doesn’t exist in all jurisdictions, which is why CC-Zero exists. If your goal is to build free software that other people can reuse, I wouldn’t recommend it.

                                                                  If your goal is to spite Google, use something like AGPL instead.

                                                                  1. 1

                                                                    WTFPL isn’t a public domain dedication. It’s an informal statement.

                                                                    1. 1

                                                                      the public domain doesn’t exist in all jurisdictions

                                                                      Which is exactly the reason cited for the creation of WTFPL. It’s basically CC-less-than-zero (and it predates CC0 by several years).

                                                                    2. 5

                                                                      … and I’ll bet doesn’t work well on non-Chrome (maybe non-Firefox?) browsers either - and thus not on Plan 9. Whereas godocs.io seems to work nicely on Netsurf.

                                                                      Edit: To my surprise, it (somewhat) does, albeit with horrendously broken layout.

                                                                      1. 7

                                                                        Can confirm (from my experience) on Firefox that pkg.go.dev has some… paper cuts, whereas on Chrome I haven’t run into any issues. godocs.io on the other hand has worked flawlessly on both and is my go to.

                                                                  1. 4

                                                                    The look is only half of the game though. I’ve never seen a cross platform UI toolkit behaving natively, and integrating with the rest of the DE. Some paradigms are just too different.

                                                                    1. 2

                                                                      It almost feels like Python was written in a deliberately slow way. Like:

                                                                      sum([w * e for (w,e) in zip(a,x)])
                                                                      

                                                                      Simply removing square brackets to avoid creation of intermediate lists in memory would speed this line up very significantly.

                                                                      And that’s not even touching on the subject of numpy: most people actually using Python for statistics would use it for such a task, because why not? So it’s not really “LispE vs. Python”, it’s “LispE vs. deliberately unoptimized pure-Python code”.

                                                                      1. 4

                                                                        For your second point, the problem with punting to a library is that the same (or similar) libraries can be used from Lisp, making the comparison pointless. I’m not sure about LispE, but Common Lisp at least has bindings for a lot of math libraries (BLAS, GSL, LAPACK, etc.).

                                                                        On the other hand, using one of those seems like overkill for the examples in the article. And if a language is so slow that any meaninful work needs to pull in third party libraries like NumPy, then that’s worth knowing, too.

                                                                        1. 2

                                                                          Simply removing square brackets to avoid creation of intermediate lists in memory would speed this line up very significantly.

                                                                          Didn’t seem to have much effect when I tried it out. If anything the code became slightly slower, 63ish ms vs 59ish on my computer. Why? No idea. The variance on either was +/- 3 ms anyway though, just from eyeballing it, so it’s a little difficult to know whether that difference was real.

                                                                          1. 1

                                                                            That’s a good observation, thanks for going through the trouble :-). It is true that on small lists generator expressions can be slower than list comprehensions. It was just something that jumped on me from cursory looking.

                                                                            1. 2

                                                                              Yeah list comprehensions are faster at a certain size, because think of the machinery to do generators. Unfortunately, I think that size is pretty large :)

                                                                          2. 2

                                                                            And that’s not even touching on the subject of numpy: most people actually using Python for statistics would use it for such a task, because why not? So it’s not really “LispE vs. Python”, it’s “LispE vs. deliberately unoptimized pure-Python code”.

                                                                            This seems like a pretty incurious reading of the article. Mine is that it’s quite explicit on exactly this question: the answer is, this algorithm was chosen as an exemplar of a sufficiently computationally complex yet still-understandable benchmark case for comparing the two languages as general compute languages. Not as a demonstration of why you should use this lisp to do your data engineering. The fact that the domain of this algorithm, in the real world, tends to revolve around some other highly optimized library is completely neither here nor there.

                                                                            1. 2

                                                                              Thank you, you are absolutely right

                                                                          1. 0

                                                                            By the way, Ubuntu Mono has been replaced with DejaVu Sans Mono on Ubuntu a few years back. I’m not sure many people use Ubuntu Mono anymore.

                                                                            1. 3

                                                                              Ubuntu Mono

                                                                              For what it’s worth, I use Ubuntu Mono on several OSs, but I don’t run Ubuntu itself. I just really like the font.

                                                                              1. 2

                                                                                I like the shape but it has an unfortunate problem of being noticeably smaller at same nominal sizes, compared to other fonts.

                                                                            1. 1

                                                                              So I guess that’s my chance to use this as drop-in-replacement for the old debouncer in notify..

                                                                              1. 1

                                                                                Every time I’m about to write something like this I’m more than a little sure that not a single soul would find it useful. It’s nice to be mistaken :-)

                                                                                P.S. I didn’t know cross-platform notify was a thing!

                                                                              1. 7

                                                                                The best any static type checking will let you do is array[float]

                                                                                Can’t Idiris at least do the “Monotonically Increasing Float” part?

                                                                                1. 9

                                                                                  It can do the whole thing, because the dependent type system is capable of end-to-end formal verification. Any FM coding language can verify properties statically, even if it’s not in the type system, so you can do this in HOL4, SPARK, Dafny, etc. It’s just balls-hard (because formal verification is balls-hard in principle).

                                                                                  1. 12

                                                                                    It can do the whole thing, because the dependent type system is capable of end-to-end formal verification.

                                                                                    This effectively moves the complexity of expressing your invariants from one place (like tests) to another (a type system extensible by arbitrary code).

                                                                                    Type systems only feel magical as long as they stay simple and declarative. Which necessarily keeps them limited.

                                                                                    1. 3

                                                                                      Type systems only feel magical as long as they stay simple and declarative.

                                                                                      I wholeheartedly agree with this statement, but I doubt you’ll find any pair of programmers that agree on what’s simple and declarative. I bet if you asked around, you’d find that any point in the entire spectrum will be picked by someone as the sweet spot.

                                                                                    2. 2

                                                                                      Last I checked most of these end-to-end FV attempts require a lot of human intervention (i.e. a human has to prove the properties and encode it in the coding language). Has that changed?

                                                                                      1. 10

                                                                                        Nope, still sucks and I still hate it

                                                                                        (It’s gotten better in the past 6 years as people got better at SMTing all the things, but you still need a lot of human intervention to get anything interesting verified)

                                                                                        1. 1

                                                                                          Lean has some cool machine-assisted tactics like suggest and library_search that will offer help to find a theorem or proposition that will complete a proof, but these are pretty brute and still often require a lot of work. They’re hit-or-miss but when they hit it’s like magic.

                                                                                    3. 9

                                                                                      Every half-serious statically typed language can create new types kilometers and miles from float. Then you won’t confuse array[kilometers] and array[miles] and possibly prevent $330 million dollars from being almost totally wasted (https://en.wikipedia.org/wiki/Mars_Climate_Orbiter).

                                                                                      That’s quite a bit better than array[float]. And you don’t need to use a research language to do it, either.

                                                                                      1. 3

                                                                                        Mars climate orbiter is a good example of why the phrase ’‘should be caught in testing” is not sufficient justification for not using additional checks and fail-safes. I assume NASA LM’s testing regimes are fairly rigorous (correct me if there is evidence to the contrary) but in some applications you need another layer of checking. Having said that I also assume that the software in question did have a type system, and it simply wasn’t used in the way you describe.

                                                                                        1. 14

                                                                                          The type system couldn’t help in this case: the data in US units (pound-seconds) which should have been in metric units (Newton-seconds) was read from a file on disk. The software reading the data from that file thought it was metric, but it wasn’t. Perhaps the file format should have included the units? But the file format as specified was only ever supposed to contain Newton-seconds, so why would they include the units?

                                                                                          Anyway, it was an interface problem, rather than a type system problem. See Mars Climate Orbiter Mishap Investigation Board Phase I Report

                                                                                          1. 3

                                                                                            In a way, the interface problem was a type system problem, but perhaps one we aren’t well equipped to solve across systems.

                                                                                            One could imagine attaching typed schemas to certain files, and writing language bindings that enforced that stored values must be strongly typed, with explicit units.

                                                                                            1. 3

                                                                                              Seems like modern type systems don’t solve this problem as c– mentioned. It is certainly true that no error checking method is 100% foolproof. My main point above was merely that testing isn’t either, and having multiple independent error checking systems is a good idea. Type systems are useful but should not be relied upon.

                                                                                              1. 1

                                                                                                Absolutely. It even goes beyond merely having multiple independent error checking systems: you have to have the processes, or organizational culture, in place to act accordingly. It’s interesting that the Mars Climate Orbiter problem was actually detected multiple times before the disaster occurred, but no-one quite took it seriously enough.

                                                                                      2. 3

                                                                                        This misses the payoff of things like type-classes, where you can get the language to provide implementations of all sorts of things. A simple example is deriving lexicographic ordering for tuples if the type of each member is ordered. As much as I enjoy editing in a sexp-aware editor, the idea that static types only constrain you is very outdated thinking.

                                                                                        1. 2

                                                                                          Even dispensing with sophisticated types, having a smart constructor that might fail at runtime but on success returns a wrapper type which guarantees it’s an “array of numbers with a mean value of such and such and a standard deviation of such and such” can help avoid many mistakes.

                                                                                        1. 2

                                                                                          The enterprise community is uniting around Bazel right now.

                                                                                          1. 3

                                                                                            Citation needed! No seriously, which enterprise community is that supposed to be?

                                                                                            1. 3

                                                                                              It means infra/tools ops people migrating between the companies and bringing their favorite tools with them.

                                                                                              Which has nothing to with whether it’s a good choice or not. Enterprise companies are notorious in copying each other approaches, so whichever gets popular at any point tends to be entrenched for some years. “Google does it” is always going to win against solitary voices saying “but it’s complicated, and there’s a better way”.

                                                                                              1. 1

                                                                                                I work in an enterprise, a pretty large one and all I see is maven and some gradle. No bazel in sight. I also have been following the project a bit for year, but never met anyone who used it (outside Google, but they have the original called blaze IIRC)

                                                                                                1. 4

                                                                                                  Hey, let me make a list of enterprise who is using Bazel and then share it with Lobste.rs

                                                                                                  There is definitely an upward trend in adoption of bazel in the last 3 years, so much that I don’t think I can adequately dropping all the names in 1 go. But if you must know, I will make a small list with some big names here who have publicly said that they are adopting bazel (not just 1-2 projects):

                                                                                                  • Google
                                                                                                  • Twitter (Java, Scala)
                                                                                                  • VMWare (Erlang, C/C++)
                                                                                                  • Adobe Cloud (Go / Docker / K8s)
                                                                                                  • Tinder (Kotlin / Java)
                                                                                                  • Pinterest (Java, Go, Python, NodeJS)
                                                                                                  • Dropbox (C, Python, Go)
                                                                                                  • Huawei (C/C++)
                                                                                                  • Lyft (iOS)
                                                                                                  • Uber (Golang)
                                                                                                  • Grab (Android)
                                                                                                  • Stripe (Ruby, Python)
                                                                                                  • Apple (Go, Java)
                                                                                                  • Robinhood
                                                                                                  • Wix
                                                                                                  • SpaceX
                                                                                                  • TwoSigma
                                                                                                  • Etsy
                                                                                                  • NVDIA (C/C++, CUDA, Go, Scala)

                                                                                                  So you can see that a large portion of FAANGMULA and equivalents unicorns are adopting Bazel. Among them, Facebook is rebuilding Bucks 2.0 and has snatched some key hires from Bazel/Build community in the past years. Microsoft MSBuild is also adopting Bazel’s RBE spec. There are also bigger companies / startups who are experimenting with Bazel on 1 or 2 smaller projects:

                                                                                                  • Redhat
                                                                                                  • Gitlab
                                                                                                  • Brave
                                                                                                  • BMW
                                                                                                  • LinkedIn
                                                                                                  • Tweag

                                                                                                  There are a number of startups / consulting companies in this space, some include members of the Blaze team leaving Google, to address the upcoming market demand:

                                                                                                  • BuildBuddy
                                                                                                  • EngFlow
                                                                                                  • FlareBuild
                                                                                                  • Aspect Dev
                                                                                                  • OasisDigital

                                                                                                  The above are not an exhaustive list, it’s what in my head after 15 mins of writing this post.

                                                                                                  1. 1
                                                                                                2. 2

                                                                                                  Google people are trying to push Bazel into everything that they touch. It’s a fine build system if you can guarantee that you will only ever want to ship things on one of the three or four platforms that Google thinks are important. Since ‘Enterprise’ is often a synonym for ‘expensive lock-in’, I don’t have any reason to doubt the original claim.

                                                                                                3. 2

                                                                                                  Do you mean Google? Or anyone else?

                                                                                                  (I’ve read that Bazel has been designed especially for building Google projects, on Google architecture, using Google style of source management)

                                                                                                  1. 1

                                                                                                    Square.

                                                                                                    1. 1

                                                                                                      As well as Twitter, Dropbox & Pinterest (all to varying degrees).

                                                                                                1. 1

                                                                                                  Still no icons on the desktop …

                                                                                                  1. 17

                                                                                                    What do you mean “still”? This is obviously a design decision, not just something they couldn’t yet find time to get to.

                                                                                                    1. 6

                                                                                                      They’re onto something. My Mac desktop is perpetually full of random disorganized files that I occasionally dump into “old” folders, creating sediment layers of garbage.

                                                                                                      1. 4

                                                                                                        It’s one of those things that either work for you or they don’t. And it looks like there’s enough people in both camps to not let there be one correct solution.

                                                                                                        My desktop is basically a temp/work/todo space with a handful of things that never get out of control. A matter of habit.

                                                                                                        1. 2

                                                                                                          Yeah, I really hate the forced no-desktop design of some DEs (GNOME & Pantheon). Why can’t they just have an option to enable/disable the desktop like in Windows? I also like to drag temporary files to my desktop and I pretty consistently tidy them up. I find not being able to use a desktop really hinders my workflow

                                                                                                          1. 1

                                                                                                            It’s one of those things that either work for you or they don’t.

                                                                                                            Funny that you say that. I used to be 100% strict “desktop is clean and minimal” for like 20 years, and only in the last few years it’s been a dump of temp things and screenshots. Only on Windows (private, games) though, where I don’t have a shell open all day like on a work machine

                                                                                                          2. 1

                                                                                                            I like to make the desktop icons uncomfortably large. It motivates me to clean them up.

                                                                                                            1. 1

                                                                                                              On the first of the month, I try to go through and clean up my desktop, downloads folder, and tabs. It’s better than before but still inconsistent.

                                                                                                          3. 2

                                                                                                            For a desktop system that seems the most stupid decision you can make …

                                                                                                            I have tried to put elementaryOS on my buddy’s laptop - he really liked it but really missed the icons on the desktop - so I installed caja and thunar to enable them by hand. The whole ‘experience’ went to trash as caja or thunar ‘desktop icons’ are not well integrated into elementaryOS DE …

                                                                                                            I have put Ubuntu Mate with ‘Cupertino’ layout to him instead and e could not be happier.

                                                                                                            I am not a ‘target’ for elementaryOS so I do not care that much but IMHO they lose a lot of ‘typical’ desktop users that have used icons on the desktop since decades.

                                                                                                            Just my $0.02 here.

                                                                                                        1. 2

                                                                                                          It’s a content-free comment, but I have to say I love this guy! He went onto the crusade I never worked up courage to do myself, apart from doing it orally among my OOP-brainwashed colleagues.

                                                                                                          1. 2

                                                                                                            Thanks. I really appreciate it. There’s a lots of negative feedback, which is to expected, so a little shout of support is nice to see.

                                                                                                            My motivation comes from the fact that I’ve seen many projects completely devastated by OOP (one could argue if it was a “Good OOP” or just “incorrect” OOP). Teams producing monstrosities where Object–relational impedance mismatch made the code insufferable to work with, the team blaming it on the ORM they used, and move on to make a rewrite using a different ORM but same OOP style, which leads them into the same place. It keeps popping up wherever I go, as OOP is still considered a blessed default methodology of some kind - spoon-fed to all the new developers. I have wrote many (mostly in free time) unenlightened-OOP projects myself in the past, so I know how it is to be bamboozled and confused by OOP. So I think people have to speak up: either steelman this stuff, and make sure OOP is understood and practice in some reasonable form, or just keep ranting on it, until it completely looses its marketing appeal.

                                                                                                            1. 12

                                                                                                              I have wrote many (mostly in free time) unenlightened-OOP projects myself in the past, so I know how it is to be bamboozled and confused by OOP. So I think people have to speak up: either steelman this stuff, and make sure OOP is understood and practice in some reasonable form, or just keep ranting on it, until it completely looses its marketing appeal.

                                                                                                              I’m sorry but I really dislike this. You’re going to continue ranting about OOP because you dislike it, and flood this site with your low-content rants? I really don’t think this kind of attitude belongs on this site. Twitter is probably a better home for this feelings-based engagement.

                                                                                                          1. 3

                                                                                                            This is one of those things you can’t solve by writing a text, no matter how smart. Both SICP and the author are right, but it’s a question of when to optimize and how much. And it only can be learned over time.

                                                                                                            1. 3

                                                                                                              I really want a button to fast-foward to the time when a) authors mostly slowed down writing those b) users figured out which ones are the most useful and c) they come by default with modern unixes. I’m fomoing real hard for not having energy to invest in acquainting myself with the new tools knowing that three quarters of them are going to be extinct in 5 years.

                                                                                                              1. 5

                                                                                                                I can relate to this, but if nobody’s writing new stuff, the platform is defunct. The best trick I’ve found for dealing with the FOMO here is to take the long view, and mostly ignore stuff until it filters into the set of long-term tools, as defined by something like “can I get a decent version of this in a stable Debian”, or passes whatever else your personal threshold is for ease of access. “It’s already installed on my OS by default” is just an extreme version of this filter with a 15 to 50 year delay on it.

                                                                                                                1. 2

                                                                                                                  oh I totally meant “available in the repos” when I said “come by default”!

                                                                                                                  1. 1

                                                                                                                    Ah, yeah - mostly the same thing, at least if one is lucky. :)

                                                                                                              1. 26

                                                                                                                The article treats Go and Rust as on equal footing when it comes to safety. This is quite wrong, unless “safety” refers only to memory safety: Rust is far safer than Go in general, for a number of reasons: algebraic data types with exhaustive pattern matching, the borrow checker for statically preventing data races, the ability to use RAII-like patterns to free resources automatically when they go out of scope, better type safety due to generics, etc.

                                                                                                                Of course, both languages are safer than using a dynamically typed language. So from the perspective of a Python programmer, it makes sense to think of Go as a safer alternative.

                                                                                                                There may be certain dimensions along which it makes sense to prefer Go over Rust, such as the learning curve, the convenience of a garbage collector, etc. But it’s important to be honest about each language’s strengths and weaknesses.

                                                                                                                1. 22

                                                                                                                  On the other hand, if you do think about memory safety, it’s not quite so clear cut. In Go, I can create a cyclic data structure without using the Unsafe package and the GC will correctly collect it for me. In Rust, I have to write custom unsafe code to both create it and clean it up. In Go I can create a DAG and the GC will correctly clean it up. In Rust, I must use unsafe (or a standard-library crate such as RC that uses unsafe internally) to create it, and clean it up.

                                                                                                                  However, in Go I can create an object with a slice field and share it between two goroutines and then have one write update it in a loop with a small slice and a large slice and the other goroutine read until it sees the base of the small slice and the length of the large slice. I now have a slice whose bounds are larger than the underlying object and I can violate all of the memory-safety invariants without writing a single line of code using the Unsafe package. In Rust, I could not do this without incorrectly implementing the Sync trait, and you cannot implement the Sync trait without unsafe code.

                                                                                                                  Go loses its memory safety guarantees if you write concurrent software. Rust loses its memory safety guarantees if you use non-trivial data structures. C++ loses its memory safety guarantees if you use pointers (or references).

                                                                                                                  1. 12

                                                                                                                    Go loses its memory safety guarantees if you write concurrent software. Rust loses its memory safety guarantees if you use non-trivial data structures. C++ loses its memory safety guarantees if you use pointers (or references).

                                                                                                                    This is fantastically succinct. Thanks, I might use this myself ;)

                                                                                                                    1. 6

                                                                                                                      In Rust, I have to write custom unsafe code to both create it and clean it up

                                                                                                                      No, you really don’t.

                                                                                                                      You can create it with no unsafe code (outside of the standard library) and no extra tracking by using Box::leak, it will just never be cleaned up.

                                                                                                                      You can create it with no unsafe code (outside of the standard library) and reference counted pointers by using Rc::new for forward pointers and Rc::downgrade for back pointers, and it will be automatically cleaned up (at the expense of adding reference counting).

                                                                                                                      You can make use of various GC and GC like schemes with no unsafe code (outside of well known libraries), the most famous of which is probably crossbeam::epoch.

                                                                                                                      You can make use of various arena datastructures to do so with no unsafe code (outside of well known libraries), provided that that form of “GC all at once at the end” fits your use case, e.g. typed arena.

                                                                                                                      1. 3

                                                                                                                        You can create it with no unsafe code (outside of the standard library)

                                                                                                                        The parenthetical here is the key part. The standard library implementations for all of the things that you describe all use unsafe.

                                                                                                                        1. 6

                                                                                                                          No, it isn’t. That’s how the entire language works, encapsulate and abstract unsafe things until they are safe. To argue otherwise is to argue that every allocation is bad, because implementing an allocator requires unsafe (and the standard library uses unsafe to do so)…

                                                                                                                          Unsafe code is not viral.

                                                                                                                          1. 1

                                                                                                                            Note also that the Rust Standard library has special dispensation for unsafe and unstable features – it can assume a particular compiler version, it can use unsafe code that would be unsound without special knowledge of the compiler, and it can compel the compiler to change in order to support what the stdlib wants to do.

                                                                                                                      2. 13

                                                                                                                        Of course, both languages are safer than using a dynamically typed language.

                                                                                                                        I wish people would stop saying that. Especially with “of course”. We can believe all we want, but there is no data supporting the idea that dynamically typed languages are inherently less safe. Again, I don’t care why you think it should be the case. First show me that it actually is, then try to hypothesize as to why.

                                                                                                                        1. 21

                                                                                                                          I often find production bugs in dynamically typed systems I work on which are due to issues that would be caught be a type checker for a modern type system (e.g., null reference errors, not handling a case of an algebraic data type when a new one is added, not handling new types of errors as the failure modes evolve over time, etc.). That’s an existence proof that having a type checker would have helped with safety. And this is in a codebase with hundreds of thousands of tests and a strict code review policy with code coverage requirements, so it wouldn’t be reasonable to attribute this to an insufficient test suite.

                                                                                                                          Very large companies are migrating their JavaScript codebases to TypeScript precisely because they want the safety of static types. Having been privy to some of those discussions, I can assure you there were made with a lot of consideration due to the enormous cost of doing so.

                                                                                                                          Going down the academic route, dependent types let you prove things that tests cannot guarantee, such as the fact that list concatenation is associative for all inputs. I personally have used Coq to prove the absence of bugs in various algorithms. As Dijkstra said, “program testing can be used to show the presence of bugs, but never to show their absence”. Types, on the other hand, actually can show the absence of bugs if you go deep enough down the rabbit hole (and know how to formally express your correctness criteria).

                                                                                                                          You don’t have to believe me, and perhaps you shouldn’t since I haven’t given you any specific numbers. There are plenty of studies, if you care to look (for example, “A Large Scale Study of Programming Languages and Code Quality in Github; Ray, B; Posnett, D; Filkov, V; Devanbu, P”). But at the same time, there are studies that claim the opposite, so for this topic I trust my personal experience more than the kind of data you’re looking for.

                                                                                                                          1. 2

                                                                                                                            Yet Go lacks many of those features, such as algebraic data types, exhaustive switches, and has both nil and default values.

                                                                                                                            1. 3

                                                                                                                              Yes, hence my original claim that Rust is far safer than Go. But Go still does have a rudimentary type system, which at least enforces a basic sense of well-formedness on programs (function arguments being the right type, references to variables/functions/fields are not typos, etc.) that might otherwise go undetected without static type checking. Since these particular types of bugs are also fairly easy to catch with tests, and since Go programs often rely on unsafe dynamic type casting (e.g., due to lack of generics), Go is not much safer than a dynamically typed language—in stark contrast to Rust. I think one could reasonably argue that Go’s type system provides negligible benefit over dynamic typing (though I might not go quite that far), but I do not consider it reasonable to claim that static types in general are not capable of adding value, based on my experience with dependent types and more sophisticated type systems in general.

                                                                                                                              1. 2

                                                                                                                                Since these particular types of bugs are also fairly easy to catch with tests, and since Go programs often rely on unsafe dynamic type casting (e.g., due to lack of generics), Go is not much safer than a dynamically typed language—in stark contrast to Rust.

                                                                                                                                But only <1% of Go code is dynamically typed, so why would you argue that it’s not much safer than a language in which 100% of code is dynamically typed? Would you equally argue that because some small amount of Rust code uses unsafe that Rust is no safer than C? These seem like pretty silly arguments to make.

                                                                                                                                In my experience writing Go and Rust (and a whole lot of Python and other languages), Go hits a sweet spot–you have significant type safety beyond which returns diminish quickly (with respect to safety, anyway). I like Rust, but I think your claims are wildly overstated.

                                                                                                                            2. 2

                                                                                                                              so for this topic I trust my personal experience more than the kind of data you’re looking for.

                                                                                                                              I wonder how much of this is us as individual programmers falling into Simpson’s Paradox. My intuition says that for large, complex systems that change infrequently, static typing is a huge boon. But that’s only some portion of the total programs being written. Scientific programmers write code that’s more about transmitting mathematical/scientific knowledge and easily changeable for experimentation. Game scripters are looking for simple on-ramps for them to change game logic. I suspect the “intuitive answer” here highly depends on the class of applications that a programmer finds themselves working on. I do think there’s an aspect of personality here, where some folks who enjoy abstraction-heavy thinking will gravitate more toward static typing and folks who enjoy more “nuts-and-bolts” thinking may gravitate toward dynamic typing. Though newer languages like Nim and Julia are really blurring the line between dynamic and static.

                                                                                                                              1. 2

                                                                                                                                Have you done Coq to prove the correctness of anything that wasn’t easy by inspection? I’ve looked at it and I’m definitely interested in the ideas (I’m working through a textbook in my spare time), but I’ve never used it to prove anything more complicated than, say, linked list concatenation or reversing.

                                                                                                                                And how do you generate the program from your code? Do you use the built-in extraction, or something else?

                                                                                                                                1. 2

                                                                                                                                  I often find production bugs in dynamically typed systems I work on that are due to things that would be caught be a type checker

                                                                                                                                  I can offer an equally useless anecdotal evidence of my own practice where bugs that would be caught by a type checked happen at a rate of about 1/50 to those caused by mis-shapen data, misunderstanding of domain complexity and plain poor testing, and when they do, they’re usually trivial to detect and fix. The only thing that tells me is that software development is complex and we are far from making sweeping statements that start from “of course”.

                                                                                                                                  Very large companies are migrating their JavaScript code bases to TypeScript for exactly for that reason.

                                                                                                                                  Sorry, the “thousand lemmings” defense won’t work here. Out whole industry has been investing countless engineer-years in OO abstractions, but then people started doing things without it and it turned out OO wasn’t the requirement for building working systems. Software development is prone to fads and over-estimations.

                                                                                                                                  Types, on the other hand, actually can show the absence of bugs

                                                                                                                                  That’s just plain wrong. Unless you mean some very specific field of software where you can write a formal specification for a program, but to this day it’s just not practical for anything that’s useful.

                                                                                                                                  1. 5

                                                                                                                                    It’s clear that many people find value in static types even if you don’t. Maybe you make fewer mistakes than the rest of us, or maybe you’re working in a domain where types don’t add as much value compared to others. But you shouldn’t try to invalidate other people’s experiences of benefitting from static types.

                                                                                                                                    they’re usually trivial to detect and fix

                                                                                                                                    I prefer to eliminate entire categories of errors without having to detect and fix them down the line when they’ve already impacted a user.

                                                                                                                                    That’s just plain wrong.

                                                                                                                                    Maybe you haven’t used formal verification before, but that doesn’t mean it isn’t used in the real world. There’s a great book series on this topic if you are interested in having a more open mind. I’ve used these kind of techniques to implement parts of a compiler that are guaranteed correct. Amazon also uses deductive techniques in multiple AWS teams (example), and there’s a decent chance you have indirectly benefitted from some of those efforts. So, my claim is not “just plain wrong”. As you alluded to, it usually doesn’t make sense to invest that much in those kinds of formal guarantees, but it’s nice that types can do that for you when you need them to.

                                                                                                                                    At this point, it seems like you aren’t interested in having a good faith discussion, with your abrasive comments like “I don’t care why you think it should be the case”, “equally useless anecdotal evidence”, and dismissing a demonstrable claim as “just plain wrong”. I think you have some good points (e.g., I completely agree about your position on OO) and could be more effective at delivering them if you didn’t seem so invested in discounting other people’s experiences.

                                                                                                                                    I respect your opinion that I should not have stated that Rust and Go are “of course” safer than dynamically typed languages. In particular, Go’s type system is so underpowered that I can see a reasonable argument that the ceremony of appeasing it without reaping the guarantees that a better type system would give makes it more difficult to build robust software than not having types at all. I certainly wouldn’t say the same for Rust, though. Rust often forces me to handle error cases that I didn’t even know were possible and would never think to test.

                                                                                                                                    1. 1

                                                                                                                                      But you shouldn’t try to invalidate other people’s experiences of benefitting from static types.

                                                                                                                                      Go’s type system is so underpowered that I can see a reasonable argument that the ceremony of appeasing it without reaping the guarantees that a better type system would give makes it more difficult to build robust software than not having types at all.

                                                                                                                                      Do you not think this is invalidating the experience of Go users who benefit from usage of the language?

                                                                                                                                      1. 1

                                                                                                                                        I said I could see it as a reasonable argument, not that I personally agree with it. I’m trying to give some validity to what isagalaev is saying and potentially meet them in the middle by acknowledging that not all type systems provide a clear benefit over dynamic typing. But I already stated my stance in my original top-level comment: that Go’s type system is still better than no type system when it comes to safety (not memory safety, but a more broad notion of safety).

                                                                                                                                        It is true, though, that Go’s type system is quite weak compared to other type systems. That’s not the same as saying that people don’t benefit from it. On the contrary, I’ve claimed the opposite—which is what started this whole discussion in the first place.

                                                                                                                                      2. 1

                                                                                                                                        abrasive comments

                                                                                                                                        Apologies on that, fwiw.

                                                                                                                                      3. 2

                                                                                                                                        […] caused by mis-shapen data, […]

                                                                                                                                        Statements like these always make me suspect the author doesn’t appreciate just how much can in fact be captured by even a relatively simple type system. “Make illegal states unrepresentable” has become something of a mantra in the Elm community in particular, and I can’t remember the last time I saw a bug in a typed FP language that was due to “mis-shappen data” get past the type checker.

                                                                                                                                        I think there’s a tendency to try to compare languages by lifting code wholesale from one language into the other, assuming it would be written more or less the same way, which is often not the case. So, if you see a something that throws TypeError, of course you assume that would get caught by a static type system. Folks who have only worked with type systems like those in Java/Go/C generally look at null/nil bugs and assume that those wouldn’t get caught, even though they’re impossible in other systems. It’s easy to point out “null isn’t a thing in this language,” but what’s a bit harder to capture is that a lot of things that aren’t obviously type errors that crop up at runtime in a dynamically typed language would likely be captured by types in a program writing with the benefit of a modern type system. Obviously it won’t if you just write a stringly-typed program, but… don’t do that, use your tools.

                                                                                                                                        1. 1

                                                                                                                                          “Make illegal states unrepresentable” has become something of a mantra in the Elm community in particular, and I can’t remember the last time I saw a bug in a typed FP language that was due to “mis-shappen data” get past the type checker.

                                                                                                                                          It’s not about illegal states. I mean code expecting a response from an HTTP call in a particular schema and getting a different one. I don’t see how this problem can be prevented at compile time. Or more subtly, getting data in a correct shape (say, an ISO-formatted time string), successfully parsing it (into an internal datetime type), but then producing the result that the user doesn’t expect because it assumes the time to be in a different time zone.

                                                                                                                                          (Also, I don’t see how making illegal states unrepresentable is in any way endemic to type-checked languages. It’s just a good architectural pattern valid everywhere.)

                                                                                                                                          1. 1

                                                                                                                                            I mean code expecting a response from an HTTP call in a particular schema and getting a different one.

                                                                                                                                            Ah, you are talking about something somewhat different then. Yes, obviously types can’t statically prove that inputs that come in at runtime will be well-formed. However, they can force you to deal with the case of a parse failure – which many languages make it really easy to forget. “Forgetting a case” is another one of those things that I think people often incorrectly assume aren’t (or can’t easily be made) type errors. It’s hard to say what you should do in that case without more context, but it makes it hard to introduce a bug by omission.

                                                                                                                                            If the bug is just that the programmer was mistaken about what the endpoint’s schema was (or the server operator changed it inappropriately), I’ll agree that just having static types in your client program does not really help that much, though it might be a bit easier to track down since the error will occur right at the point of parsing, rather than some time later when somebody tries to use the value.

                                                                                                                                            That said, I’ll point to stuff like protobuf, capnproto, and even swagger as things that are trying to bridge this gap to some degree – there is still an opportunity to just assign the entirely wrong schema to an endpoint, but they narrow the space over which that can happen substantially; once you’ve got the right schema rigged up the programmer is unlikely to get the shape of the data wrong, as that’s just defined in the schema.

                                                                                                                                            Or more subtly, getting data in a correct shape (say, an ISO-formatted time string), successfully parsing it (into an internal datetime type), but then producing the result that the user doesn’t expect because it assumes the time to be in a different time zone.

                                                                                                                                            Dealing with fiddly distinctions like this is something types are really great at. I have some good points of comparison with date/time stuff, as I’ve worked on projects where this stuff is core to the business logic in both Python and Haskell. Having a type system for it in Haskell has been a godsend, and I wish I’d had one available when doing the prior project.

                                                                                                                                            Somewhat simplified for presentation (but with the essentials in-tact), the Haskell codebase has some types:

                                                                                                                                            • A date & time with an attached (possibly arbitrary) time zone, “ZonedDateTime”
                                                                                                                                            • A date & time in “local time,” where the timezone is implied by context somehow. “LocalDateTime”

                                                                                                                                            As an example of where this is useful: the code that renders the user’s feed of events in order expects a list of LocalDateTime values, so if you try to pass it the datetime with some arbitrary timezone, you’ll get a type error. Instead, there’s a function timeInZone which takes a ZonedDateTime and a TimeZone, and translates it to a LocalDateTime with the provided timezone implied. So in order to get an event into a users feed, you need to run it through this conversion function, or it won’t type check.

                                                                                                                                            (Also, I don’t see how making illegal states unrepresentable is in any way endemic to type-checked languages. It’s just a good architectural pattern valid everywhere.)

                                                                                                                                            It’s a lot easier to do it when you can actually dictate what the possible values of something are; if in the event of a bug a variable could have any arbitrary value, then your options for enforcing invariants on the data are much more limited. You can put asserts everywhere, but having static types is much much nicer.

                                                                                                                                    2. 6

                                                                                                                                      Also, people talk about “dynamic languages” like they’re all the same, while they are often as different as C and Rust, if not more.

                                                                                                                                      Writing safe Javascript is an impossible nightmare..

                                                                                                                                      Writing safe Python is easy and fun. I do think it would be safer with a good type-system, but at the same time, a shorter and more idiomatic code (where you don’t have to fight with the compiler) brings its own sort of safety and comfort.

                                                                                                                                      1. 4

                                                                                                                                        Writing safe Python is easy and fun.

                                                                                                                                        Python has its own share of footguns, such as passing strings somewhere where bytes are expected, or new, unhandled exceptions being added to libraries you’re calling.

                                                                                                                                        Mypy doesn’t completely protect you. IME, many errors occur at the seams of typed/untyped contexts. Which is not surprising, but it is a downside of an after-the-fact, optional type checker.

                                                                                                                                        1. 1

                                                                                                                                          Yeah, and mypy has far less coverage for third-party packages than TypeScript. IME when I use TS I’m surprised if I don’t find a type package, whereas with mypy I’m surprised if I do.

                                                                                                                                          1. 1

                                                                                                                                            About one in twenty times I do see a types package in DefinitelyTyped which is very incorrect. Almost always only with libraries with small user bases.

                                                                                                                                            1. 1

                                                                                                                                              Incorrect as in “this parameter is a number but is typed as a string”, or too loose/too restrictive?

                                                                                                                                              1. 1

                                                                                                                                                Varies. I’ve seen symbols typed as strings, core functionality missing, functions that couldn’t be called without “as any”.

                                                                                                                                          2. 1

                                                                                                                                            Python isn’t perfect, but unhandled exceptions can happen in C++ too..

                                                                                                                                            It doesn’t have to be after the fact. You can check types at run-time, and there are libraries that will help you with that. It comes at a slight performance cost, of course (but if that matters, why are you using Python?), but then you gain the ability to implement much more sophisticated checks, like contracts, or dependent types.

                                                                                                                                            Anyway, at least personally, type errors are rarely the real challenge when writing software.

                                                                                                                                        2. 3

                                                                                                                                          there is no data supporting the idea that dynamically typed languages are inherently less safe

                                                                                                                                          If we use the Gary’s Types document for the definition of a dynamically typed language, I am able to find some research:

                                                                                                                                          I do agree that this amount of research is far away from us being able to say “of course”.

                                                                                                                                          1. 3

                                                                                                                                            Not gonna take a position here on the actual static vs dynamic debate, but the second paper you linked is deeply flawed. I wrote a bit about the paper, and the drama around it, here: https://www.hillelwayne.com/post/this-is-how-science-happens/

                                                                                                                                            1. 1

                                                                                                                                              Awesome! Thank you for sharing.

                                                                                                                                          2. 2

                                                                                                                                            I think it stands to reason that statically typed languages are safer than dynamically typed languages. I’m vaguely aware of some studies from <2005 that compared C++ to Python or some such and found no significant difference in bugs, but I can’t imagine those findings would hold for modern mainstream statically typed languages (perhaps not even C++>=11). Personally I have extensive experience with a wide array of languages and my experience suggests that static typing is unambiguously better than dynamic typing; I hear the same thing from so many other people and indeed even the Python maintainers–including GVR himself–are compelled. Experience aside, it also stands to reason that languages in which entire classes of errors are impossible would have fewer errors in total.

                                                                                                                                            So while perhaps there isn’t conclusive data one way or the other, experience and reason seem to suggest that one is better than the other.

                                                                                                                                            1. 1

                                                                                                                                              Experience aside, it also stands to reason that languages in which entire classes of errors are impossible would have fewer errors in total.

                                                                                                                                              What this (very common) argument is missing is that a) those errors are not as frequent as the words “entire classes of errors” may lead to believe, that b) testing covers many of those errors better (by testing for correct values you’re getting correct types for free), and that c) instrumenting your code with types isn’t free, you get bloated and more rigid code that may lead to more serious errors in modeling your problem domain.

                                                                                                                                              1. 4

                                                                                                                                                I’ve personally found that relying on a decent type system makes my code more flexible, as in easier to refactor, because I can rely on the type system to enforce that all changes are valid.

                                                                                                                                                1. 1

                                                                                                                                                  I’ve developed and operated Python services in production for about a decade. Type errors were the most common kind of errors we would encounter by a wide margin. We were very diligent about writing tests, but inevitably we would miss cases. Some of these were “this function is literally just an entry point that unmarshals JSON and passes it into a library function… I don’t need a test” but they would forget to await the library function.

                                                                                                                                                  Moreover, how do you reconcile “annotating your code takes too much time and makes your code too inflexible, but writing tests is good value for money”? Am I misunderstanding your argument? Note also that tests are useful for lots of other things, like documentation, optimizations, refactoring, IDE autocomplete, etc.

                                                                                                                                                  1. 1

                                                                                                                                                    Moreover, how do you reconcile “annotating your code takes too much time and makes your code too inflexible, but writing tests is good value for money”?

                                                                                                                                                    Types are not a replacement for tests, you have to write tests anyway. And they are good value for money, which is one of the few things we actually know about software development. So the proposition I’m making is that if you write good tests they should cover everything a type checker would. Because, essentially, if you check all your values are correct then it necessarily implies that types of those values are correct too (or at least, they work in the same way).

                                                                                                                                                    Now, to your point about “but inevitably we would miss cases” — I’m very much aware of this problem. I blame the fact that people write tests in horribly complicated ways, get burnt out, never achieve full test coverage and then turn to type checking to have at least some basic guarantees for the code that’s not covered. I’m not happy with this, I wish people would write better tests.

                                                                                                                                                    You example with a test for a trivial end point is very telling in this regard. If it’s trivial, then writing a test for it should also be trivial too, so why not?

                                                                                                                                                    1. 1

                                                                                                                                                      I disagree. In my extensive experience with Python and Go (15 and 10 years, respectively), Go’s type system grants me a lot more confidence even with a fraction of the tests of a Python code base. In other words, a type system absolutely is a replacement for a whole lot of tests.

                                                                                                                                                      I specifically disagree that checking for a value guarantees that the logic is correct (type systems aren’t about making sure the type of the value is correct, but that the logic for getting the value is sound).

                                                                                                                                                      While 100% test coverage would make me pretty confident in a code base, why burn out your engineers in pursuit of it when a test system would reduce the testing load significantly?

                                                                                                                                                      With respect to “trivial code thus trivial tests”, I think this is untrue. Writing “await foo(bar, baz)” is trivial. Figuring out how to test that is still cognitively burdensome, and cognition aside it’s many times the boilerplate.

                                                                                                                                                      Lastly, people rarely discuss how static type systems make it harder to write certain kinds of bad code than dynamic type systems. For example, a function whose return type varies depending on the value of some parameter. The typical response from dynamic typing enthusiasts is that this is just bad code and bad Go code exists too, which is true but these kinds of bad code basically can’t exist in idiomatic Go code and they are absolutely pedestrian in Python and JavaScript.

                                                                                                                                                      At a certain point, you just have to get a lot of experience working with both systems in order to realize that the difference is really quite stark (even just the amount of documentation you get out of the box from type annotations, and the assurance that that documentation is correct and up to date).

                                                                                                                                                      1. 1

                                                                                                                                                        I specifically disagree that checking for a value guarantees that the logic is correct

                                                                                                                                                        I didn’t say anything about the whole logic. I said if your values are correct then it necessarily implies your types are correct. Specifically, if you have a test that does:

                                                                                                                                                        config = parse_config(filename)
                                                                                                                                                        assert config['key'] == 'value'
                                                                                                                                                        

                                                                                                                                                        Then it means that parse_config got a correct value in filename that it could use to open and parse the config file. In which case it also means filename was of correct type: a string, or a Path or whatever the language’s stdlib could use in open(). That’s it, nothing philosophical here.

                                                                                                                                                        While 100% test coverage would make me pretty confident in a code base, why burn out your engineers in pursuit of it

                                                                                                                                                        Let me reiterate: if achieving 100% test coverage feels like a burn-out, you’re doing it wrong. It’s actually not even hard, especially in a dynamic language where you don’t have to dependency-inject everything. I’m not just fantasizing here, that’s what I did in my last three or four code bases whenever I was able to sell people on the idea. There’s this whole ethos of it being somehow impossibly hard which in many applications just doesn’t hold up.

                                                                                                                                                        1. 1

                                                                                                                                                          Some more :-)

                                                                                                                                                          Writing “await foo(bar, baz)” is trivial. Figuring out how to test that is still cognitively burdensome, and cognition aside it’s many times the boilerplate.

                                                                                                                                                          Huh? The tooling is out there, you don’t have to invent anything: https://pypi.org/project/pytest-asyncio/

                                                                                                                                                          @pytest.mark.asyncio
                                                                                                                                                          async def test_some_asyncio_code():
                                                                                                                                                              res = await library.do_something()
                                                                                                                                                              assert b'expected result' == res
                                                                                                                                                          

                                                                                                                                                          At a certain point, you just have to get a lot of experience working with both systems in order to realize that the difference is really quite stark

                                                                                                                                                          Just to clarify here, I mostly work in Python, but I also work extensively in JavaScript, TypeScript, Kotlin and Rust (much less in the latter than I would like). And my experience tells me that types is not the most significant feature that makes a language safe (for whatever value of “safe”). It is also subjective. I do absolutely trust you that you find working in Go more comfortable, but it’s important to understand that the feeling doesn’t have to be universal. I would hate to have to program in Go, even though it’s a simple language.

                                                                                                                                                2. 1

                                                                                                                                                  Genuine question: how could it be shown? You would need at least two projects of similar scope, in similar areas, written by programmers of similar skill (which is hard to evaluate on its own) and similar level of understanding of the problem area (which means that rewrites of the same code base can’t count), differing only in their choice of static/dynamic typing. How could such a research be possible?

                                                                                                                                                  More generally: is there any solid research about which languages are better? Is there any data that programs written is the assembly language are more or less error-prone than those written in Python? This should be intuitively obvious, but is there data? I tried to find anything at all, but only discovered half-baked “studies” that don’t control for either the programmer experience or the complexity of the problem area.

                                                                                                                                                  My point is, how can we do better than a bunch of anecdotes here?

                                                                                                                                                  1. 3

                                                                                                                                                    Right, I think one can admit that, as a field, we’re not in amazing shape epistemologically, but we are still left having to actually make decisions in our day to day, so not having opinions isn’t really an option – we’re stuck going on experience and anecdote, but that’s not quite the same as having no information. It’d be nice to have conclusive studies. Unfortunately, all I’ve got is a career’s worth of anecdata.

                                                                                                                                                    I have no idea how to study this kind of thing.

                                                                                                                                                    1. 3

                                                                                                                                                      Yep, this was exactly my point: when someone asks for studies that conclusively show that X is better than Y in this context, I think that they are asking for too much and people are justified in saying “of course” even in the abscence of rock-solid evidence.

                                                                                                                                                    2. 2

                                                                                                                                                      I would go so far as to argue that, often, when people insist on data for something like this, they are actually being disingenuous. If you insist on (quantitative) data when you know that none exist or are likely to exist in the future, then you are actually just saying that you want to unquestioningly maintain the status quo.

                                                                                                                                                      1. 2

                                                                                                                                                        I would go so far as to argue that, often, when people insist on data for something like this, they are actually being disingenuous.

                                                                                                                                                        … Why would it be disingenuous to ask for data? This just sounds absurd. Moreover there really isn’t consensus around this topic. Take a look at Static v. dynamic languages literature review, this has been an ongoing topic of discussion and there still isn’t a conclusion either way. Regardless this perspective frightens me. It sounds a lot like “I have an opinion and data is hard so I’m going to call you disingenuous for disagreeing with me.” This isn’t the way to make good decisions or to tolerate nuanced opinions.

                                                                                                                                                        “The problem with any ideology is it gives the answer before you look at the the evidence. So you have to mold the evidence to get the answer you’ve already decided you’ve got to have.” – Bill Clinton

                                                                                                                                                        1. 2

                                                                                                                                                          Why would it be disingenuous to ask for data?

                                                                                                                                                          It isn’t, and I didn’t say that it was.

                                                                                                                                                          I said it was potentially disingenuous to insist on (quantitative) data as a prerequisite for having a discussion. If good data don’t exist, then refusing to talk about something until good data do exist is just another way of defending whatever the status quo is. What it basically says is that regardless of how we made the current decision (presumably without data, since data don’t exist), the decision cannot be changed without data.

                                                                                                                                                          I’m honestly not sure how you came up with that interpretation based on what I wrote. I didn’t even say which side of the issue I was on.

                                                                                                                                                          Edit: you can also substitute “inconclusive data” for “no data”.

                                                                                                                                                          1. 2

                                                                                                                                                            I’m honestly not sure how you came up with that interpretation based on what I wrote. I didn’t even say which side of the issue I was on.

                                                                                                                                                            I think this is a difference between our interpretations on “insist”. I tend to read “insist” as an earnest suggestion, not a hard prerequisite, so that’s where my my disagreement came from. I didn’t mean to imply anything about which side you were on since that’s immaterial to my point really. I agree that categorically refusing to discuss without sufficient data is a bit irresponsible since in real life humans are often forced to make decisions without appropriate evidence.

                                                                                                                                                            If good data don’t exist, then refusing to talk about something until good data do exist is just another way of defending whatever the status quo is.

                                                                                                                                                            My thinking here is, at what point does this become a useless thought exercise? Static typing isn’t new and is gaining ground in several languages. There’s already a “programmer personality” identity based around static typing and “healthy” Twitter communities of folks who bemoan static or dynamic languages. At some point, the programming community at large gains nothing by having more talking heads philosophizing about where and why they see bugs. You can take a cursory search on the internet and see folks advocating for pretty much any point on the spectrum of this debate. To me this discussion (not this Lobsters thread, but the greater discussion as a whole) seems to have reached the point where it’s useless to proceed without data because there’s no consensus around which point on the spectrum of static/dynamic does actually lead to fewer (if any point does at all) bugs. And if more philosophizing doesn’t help us arrive at a conclusion, it really boils down to the same thing: your personal feelings and experiences, in which case the discussion is more of a form of socializing than a form of actual discussion. In other words, without data, this discussion trends more toward bikeshedding than actually answering the question under discussion.

                                                                                                                                                            1. 2

                                                                                                                                                              In other words, without data, this discussion trends more toward bikeshedding than actually answering the question under discussion.

                                                                                                                                                              That’s fair. I agree that this particular topic has been pretty well discussed to death.

                                                                                                                                                              1. 1

                                                                                                                                                                there’s no consensus around which point on the spectrum of static/dynamic does actually lead to fewer (if any point does at all) bugs

                                                                                                                                                                I think consensus is emerging slowly—static languages seem to have grown more popular in the last decade to the extent that many JavaScript developers are converting to TypeScript, Python developers are embracing Mypy, most (all?) of the most popular new languages of the last 10-15 years have been statically typed (the Go community in particular seems to consist of a lot of former Python and Ruby devs), etc. On the other hand, I scarcely if ever hear about people switching to dynamically typed languages (once upon a time this was common, when the popular static offerings were C, C++, and Java). It’s possible that this emerging consensus is just a fad, but things do seem to be converging.

                                                                                                                                                            2. 1

                                                                                                                                                              I suspect the problem is that this question is enormously multivariate (skill of developers, development methodology, testing effort to find bugs, different language features, readability of the language, etc).

                                                                                                                                                              It’s entirely likely that we have been studying this for a long time and yet the variable space is so large that we’ve hardly scratched it at all. And then some people come along and interpret this lack of conclusive data as “well, static and dynamic must be roughly equal” which seems strictly more périlleuse than formulating one’s own opinion based on extensive experience with both type systems.

                                                                                                                                                              I don’t think your Clinton quote applies because we’re talking about forming an opinion based on “a career’s worth of anecdata”, not letting an ideology influence one’s opinion. Everyone admits that anecdata is not as nice as conclusive empirical data, but we don’t have any of the latter and we have lots of the former and it seems to point in a single direction. In this case the ideological take would be someone who forms an opinion based on lack of evidence and lack of subjective experience with both systems.

                                                                                                                                                          2. 1

                                                                                                                                                            Genuine question: how could it be shown? You would need at least two projects of similar scope, in similar areas, written by programmers of similar skill (which is hard to evaluate on its own) and similar level of understanding of the problem area (which means that rewrites of the same code base can’t count), differing only in their choice of static/dynamic typing. How could such a research be possible?

                                                                                                                                                            Remember that “project scope”, “project area” (systems, web, database, etc), “license status” (GPLv3, proprietary, etc) are all dimensions for a project that can be recorded and analyzed. There is a rich literature of statistical methods to compensate for certain dimensions being over or underrepresented, and for compensating for incomplete data. If we can come up with a metric for being error-prone (which is difficult, so perhaps we need multiple metrics (NB: Code complexity metrics are a good look here to look at the challenges)), and we can faithfully record other dimensions of projects, we can try to rigorously answer this question. The big barriers here usually involve data siloing (proprietary projects rarely share relevant data about their projects) and just manpower (how many developers, especially of open source projects, really have the time to also gather stats about their contributors and their bugs when they can barely hold together the project itself in their generous free time, or can get approval in a proprietary project).

                                                                                                                                                            That said there’s this stubborn “philosopher-programmer” culture in programming circles that doesn’t seem particularly interested in epistemological work which also often muddles the conversation, especially if the topic under discussion has a lot of strong opinions and zealotry involved.

                                                                                                                                                            1. 1

                                                                                                                                                              The short answer is, I don’t know. I had some ideas though. Like an experiment where you solicit a public participation from a bunch of people to write from scratch something not complicated, and yet not trivial. Like, I don’t know, a simple working chat client for an existing reference server implementation. You don’t impose any restriction: people could work in any language, use libraries, etc. Time capped by, say, a couple of weeks. And then you independently verify the results and determine defects of any sort. And then you look at correlations: does number of defects correlate with dynamic/static nature? Programmer’s experience? Programmer’s experience with a language? Amount of active working hours? Something else?

                                                                                                                                                              My hypothesis is that we can’t actually evaluate a language in a vacuum. Instead a language+programmer is a actually an atomic unit of evaluation. Like in auto racing you can’t say which driver is the best (although it’s people’s favorite pastime), you can only talk about a driver in a particular car driving on particular tracks.

                                                                                                                                                              1. 2

                                                                                                                                                                There is a 1994 paper called “Haskell vs. Ada vs. C++ an Experiment in Software Prototyping Productivity” which doesn’t touch on static vs dynamic typing, but, I think, follows about the same format as what you’re proposing, right? That paper is widely disliked due to its methodological problems. An example of its discussion: https://news.ycombinator.com/item?id=14267882

                                                                                                                                                            2. 1

                                                                                                                                                              We can believe all we want, but there is no data supporting the idea that dynamically typed languages are inherently less safe

                                                                                                                                                              I think that some people tend to use the term “safe” to mean both “memory-safe” and “bug-resistant”, whereas others would use the term “safe” to refer to “memory-safe” only.

                                                                                                                                                              I can quite believe that applications written in dynamically-typed languages might be vulnerable to a whole class of bugs that aren’t present in their statically-typed equivalents because, unless you’re very careful, type coercion can silently get you in ways you don’t expect. You could write an entire book on the many mysterious ways of the JavaScript type system, for example.

                                                                                                                                                              That said, these aren’t really bugs that make the program unsafe if you’re talking about memory safety. It’s not terribly likely that these sorts of bugs are going to allow unsafe memory accesses or cause buffer overflows or anything of the sort.

                                                                                                                                                              1. 2

                                                                                                                                                                applications written in dynamically-typed languages might be vulnerable to a whole class of bugs [ … ] , type coercion can silently get you in ways you don’t expect.

                                                                                                                                                                Dynamic languages is not just JavaScript. For example Python and Clojure don’t do type coercion, nor do they silently swallow access to non-existing names and attributes.

                                                                                                                                                            3. 4

                                                                                                                                                              Of course, both languages are safer than using a dynamically typed language. So from the perspective of a Python programmer, it makes sense to think of Go as a safer alternative.

                                                                                                                                                              Returns also diminish. Using Go rather than Python will probably reduce type errors by 95% while Rust would reduce them by 99%. And that additional 4% of type errors may not be worth the hit to productivity (yes, I assert that Go is quite a lot more productive than Rust for most applications, even though I am a fan of both languages). Note also that these are just type errors; there are lots of errors which neither Rust nor Go can protect against.

                                                                                                                                                              1. 5

                                                                                                                                                                there are lots of errors which neither Rust nor Go can protect against.

                                                                                                                                                                “How does your language handle null references?”

                                                                                                                                                                Prohibited by the type system at compile time.

                                                                                                                                                                “Nice, nice. What about out-of-bounds array accesses?”

                                                                                                                                                                Sometimes detectable even at compile time and at any rate detected and safely handled at runtime.

                                                                                                                                                                “Wonderful. So obviously then if your allocator reports that you’ve run out of memory…”

                                                                                                                                                                Instant, unrecoverable crash, yes.

                                                                                                                                                                1. 2

                                                                                                                                                                  I’m not sure how that relates to my “there are lots of errors which neither Rust nor Go can protect against.” statement that you’re quoting. Yes, those are categories of errors that Rust protects against and Go does not, but there are still lots of other errors that neither language protect against.

                                                                                                                                                                  1. 1

                                                                                                                                                                    My point is that one of the errors that neither protect you against is out-of-memory errors, which has always baffled me. Rust doesn’t even panic (which could be recovered), but aborts.

                                                                                                                                                                    OOM is much more often treated as a situation where it’s seemingly okay to absolutely crash compared to other resource-exhaustion situations (nobody would be like “oh the disk is full, let’s just crash and not even attempt to let the programmer deal with it”).

                                                                                                                                                                    1. 2

                                                                                                                                                                      I don’t know the rationale for this in Rust, but I’m aware that there’s been some discussion of this in the C++ standards committee. Gracefully handling out-of-memory conditions sounds really useful, but there are two problems:

                                                                                                                                                                      • In several decades of the C++ specification defining exception behaviour for operator new exhausting memory and longer for C defining malloc as returning NULL when allocation fails, there are no examples of large-scale systems outside of the embedded / kernel space that gracefully handle memory exhaustion in all places where it can occur. Kernels generally don’t use the standard may-fail APIs and instead use two kinds of allocations, those that may block and those that may fail, with the vast majority of uses being the ones that can block.
                                                                                                                                                                      • Most *NIX systems deterministically report errors if you exhaust your address space (which is not easy on a 64-bit system) but don’t fail on out-of-memory conditions at the allocation point. They will happily report that they’ve allocated memory but then fault when you try to write to it.

                                                                                                                                                                      If you do get an out-of-memory condition, what do you do? If you’re disciplined and writing a very low-level system, then you do all of your allocation up-front and report failure before you’ve tried any processing. For anything in userspace, you typically need to do a load of cleanup, which may itself trigger allocation.

                                                                                                                                                                      In general, the set of things for which it is possible to gracefully handle allocation failure are so distinct from everyday programming that it’s difficult to provide a generic mechanism that works for both. This is what malloc(3) and malloc(9) are such different APIs.

                                                                                                                                                                      1. 2

                                                                                                                                                                        Part of the trouble is that in most languages, it’s really hard to actually do much of anything without further allocation. Especially in languages where allocation can happen implicitly, this really does seem like an “above the program’s pay grade” kind of thing.

                                                                                                                                                                        That said, Rust is decidedly not one of those languages; this is an API design choice, and it has indeed often felt like an odd one to me.

                                                                                                                                                              1. 67

                                                                                                                                                                I don’t understand how heat maps are used as a measuring tool, it seems pretty useless on its own. If something is little clicked, does it mean people don’t need the feature or people don’t like how it’s implemented? Or how do you know if people would really like something that’s not there to begin with?

                                                                                                                                                                It reminds about the Feed icon debacle: it’s been neglected for years and fell out of active use, which lead Mozilla to say “oh look, people don’t need the Feed icon, let’s move it away from the toolbar”. And then after a couple of versions they said “oh look, even less people use the Feed functionality, let’s remove it altogether”. Every time I see a click heatmap as a means to drive UI decisions I can’t shake the feeling that it’s only used to rationalize arbitrary product choices already made.

                                                                                                                                                                (P.S. I’ve been using Firefox since it was called Netscape and never understood why so many people left for Chrome, so no, I’m not just a random hater.)

                                                                                                                                                                1. 11

                                                                                                                                                                  Yeah, reminds me of some old Spiderman game where you could “charge” your jump to jump higher. They removed the visible charge meter in a sequel but kept the functionality, then removed the functionality in the sequel after that because nobody was using it (because newcomers didn’t know it was there, because there was no visible indication of it!).

                                                                                                                                                                  1. 8

                                                                                                                                                                    It’s particularly annoying that the really cool things, which might actually have a positive impact for everyone – if not now, at least in a later release – are buried at the end of the announcement. Meanwhile, some of the things gathered through metrics would be hilarious were it not for the pretentious marketing language:

                                                                                                                                                                    There are many ways to get to your preferences and settings, and we found that the two most popular ways were: 1) the hamburger menu – button on the far right with three equal horizontal lines – and 2) the right-click menu.

                                                                                                                                                                    Okay, first off, this is why you should proofread/fact-check even the PR and marketing boilerplate: there’s no way to get to your preferences and settings through the right-click menu. Not in a default state at least, maybe you can customize the menu to include these items but somehow I doubt that’s what’s happening here…

                                                                                                                                                                    Anyway, assuming “get to your preferences and settings” should’ve actually been “do things with the browser”: the “meatball” menu icon has no indication that it’s a menu, and a fourth way – the old-style menu bar – is hidden by default on two of the three desktop platforms Firefox supports, and isn’t even available on mobile. If you leave out the menubar through sheer common sense, you can skip the metrics altogether, a fair dice throw gets you 66% accuracy.

                                                                                                                                                                    People love or intuitively believe what they need is in the right click menu.

                                                                                                                                                                    I bet they’ll get the answer to this dilemma if they:

                                                                                                                                                                    • Look at the frequency of use for the “Copy” item in the right-click menu, and
                                                                                                                                                                    • For a second-order feature, if they break down right-click menu use by input device type and screen size

                                                                                                                                                                    And I bet the answer has nothing to do with love or intuition ;-).

                                                                                                                                                                    I have also divined in the data that the frequency of use for the right-click menu will further increase. The advanced machine learning algorithms I have employed to make this prediction consist of the realisation that one menu is gone, and (at least the screenshots show) that the Copy item is now only available in the right-click menu.

                                                                                                                                                                    Out of those 17 billion clicks, there were three major areas within the browser they visited:

                                                                                                                                                                    A fourth is mentioned in addition to the three in the list and, as one would expect, these four (out of… five?) areas are: the three areas with the most clickable widgets, plus the one you have to click in order to get to a new website (i.e. the navigation bar).

                                                                                                                                                                    1. 12

                                                                                                                                                                      They use their UX experts & measurements to rationalize decisions done to make Firefox more attractive to (new) users as they claim, but … when do we actually see the results?

                                                                                                                                                                      The market share has kept falling for years, whatever they claim to be doing, it is exceedingly obvious that they are unable to deliver.

                                                                                                                                                                      Looking back, the only thing I remember Mozilla doing in the last 10 years is

                                                                                                                                                                      • a constant erosion of trust
                                                                                                                                                                      • making people’s lives miserable
                                                                                                                                                                      • running promising projects into the ground at full speed

                                                                                                                                                                      I would be less bitter about it if Mozilla peeps wouldn’t be so obnoxiously arrogant about it.


                                                                                                                                                                      Isn’t this article pretty off-topic, considering how many stories are removed for being “business analysis”?

                                                                                                                                                                      This is pretty much “company losing users posts this quarter’s effort to attract new users by pissing off existing ones”.

                                                                                                                                                                      1. 14

                                                                                                                                                                        The whole UI development strategy seems to be upside down: Firefox has been hermorrhaging users for years, at a rate that the UI “improvements” have, at best, not influenced much, to the point where a good chunk of the browser “market” consists of former Firefox users.

                                                                                                                                                                        Instead of trying to get the old users back, Firefox is trying to appeal to a hypothetical “new user” who is technically illiterate to the point of being confused by too many buttons, but somehow cares about tracking through 3rd-party cookies and has hundreds of tabs open.

                                                                                                                                                                        The result is a cheap Chrome knock-off that’s not appealing to anyone who is already using Chrome, alienates a good part of their remaining user base who specifically want a browser that’s not like Chrome, and pushes the few remaining Firefox users who don’t specifically care about a particular browser further towards Chrome (tl;dr if I’m gonna use a Chrome-like thing, I might as well use the real deal). It’s not getting anyone back, and it keeps pushing people away at the same time.

                                                                                                                                                                        1. 16

                                                                                                                                                                          The fallacy of Firefox, and quite a few other projects and products, seems to be:

                                                                                                                                                                          1. Project X is more popular than us.
                                                                                                                                                                          2. Project X does Y.
                                                                                                                                                                          3. Therefore, we must do Y.

                                                                                                                                                                          The fallacy is that a lot of people are using your software is exactly because it’s not X and does Z instead of Y.

                                                                                                                                                                          It also assumes that the popularity is because of Y, which may be the case but may also not be the case.

                                                                                                                                                                          1. 3

                                                                                                                                                                            You’re not gonna win current users away from X by doing what X does, unless you do it much cheaper (not an option), or 10x better (hard to see how you could do more of chrome better than chrome).

                                                                                                                                                                            1. 1

                                                                                                                                                                              You might; however, stop users from switching to X by doing what X does, even if you don’t do it quite as well.

                                                                                                                                                                          2. 4

                                                                                                                                                                            The fundamental problem with Firefox is: It’s just slow. Slower than Chrome for almost everything. Slower at games (seriously, its canvas performance is really bad), slower at interacting with big apps like Google Docs, less smooth scrolling, even more latency between you hit a key on the keyboard and the letter shows up in the URL bar. This stuff can’t be solved with UI design changes.

                                                                                                                                                                            1. 3

                                                                                                                                                                              Well, but there are reasons why it’s slow - and at least one good one.

                                                                                                                                                                              Most notably, because Firefox makes an intentionally different implementation trade-off than Chrome. Mozilla prioritizes lower memory usage in FF, while Google prioritizes lower latency/greater speed.

                                                                                                                                                                              (I don’t have a citation on me at the moment, but I can dig one up later if anyone doesn’t believe me)

                                                                                                                                                                              That’s partially why you see so many Linux users complaining about Chrome’s memory usage.

                                                                                                                                                                              These people are getting exactly what they asked for, and in an age where low CPU usage is king (slow mobile processors, limited battery life, more junk shoved into web applications, and plentiful RAM for people who exercise discipline and only do one thing at once), Chrome’s tradeoff appears to be the better one. (yes, obviously that’s not the only reason that people use Chrome, but I do see people noticing it and citing it as a reason)

                                                                                                                                                                              1. 2

                                                                                                                                                                                I rarely use Google Docs; basically just when someone sends me some Office or Spreadsheet that I really need to read. It’s easiest to just import that in Google Docs; I never use this kind of software myself and this happens so infrequently that I can’t be bothered to install LibreOffice (my internet isn’t too fast, and downloading all updates for it takes a while and not worth it for the one time a year I need it). But every time it’s a frustrating experience as it’s just so darn slow. Actually, maybe it would be faster to just install LibreOffice.

                                                                                                                                                                                I haven’t used Slack in almost two years, but before this it was sometimes so slow in Firefox it was ridiculous. Latency when typing could be in the hundreds or thousands of ms. It felt like typing over a slow ssh connection with packet loss.

                                                                                                                                                                                CPU vs. memory is a real trade-off with a lot of various possible ways to do this and it’s a hard problem. But it doesn’t change that the end result is that for me, as a user, Firefox is sometimes so slow to the point of being unusable. If I had a job where they used Slack then this would be a problem as I wouldn’t be able to use Firefox (unless it’s fixed now, I don’t know if it is) and I don’t really fancy having multiple windows.

                                                                                                                                                                                That being said, I still feel Firefox gives a better experience overall. In most regular use it’s more than fast enough; it’s just a few exceptions where it’s so slow.

                                                                                                                                                                                1. 1

                                                                                                                                                                                  That being said, I still feel Firefox gives a better experience overall. In most regular use it’s more than fast enough; it’s just a few exceptions where it’s so slow.

                                                                                                                                                                                  I agree. I absolutely prefer Firefox to Chrome - it’s generally a better browser with a much better add-on ecosystem (Tree Style Tabs, Container Tabs, non-crippled uBlock Origin) and isn’t designed to allow Google to advertise to you. My experience with it is significantly better than with Chome.

                                                                                                                                                                                  It’s because I like Firefox so much that I’m so furious about this poor design tradeoff.

                                                                                                                                                                                  (also, while it contributes, I don’t blame all of my slowdowns on Firefox’s design - there are many cases where it’s crippled by Google introducing some new web “standard” that sites started using before Firefox could catch up (most famously, the ShaddowDOM v0 scandal with YouTube))

                                                                                                                                                                                2. 1

                                                                                                                                                                                  I don’t have a citation on me at the moment, but I can dig one up later if anyone doesn’t believe me

                                                                                                                                                                                  I’m interested in your citations :)

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    Here’s one about Google explicitly trading off memory for CPU that I found on the spot: https://tech.slashdot.org/story/20/07/20/0355210/google-will-disable-microsofts-ram-saving-feature-for-chrome-in-windows-10

                                                                                                                                                                            2. 4

                                                                                                                                                                              I remember more things from Mozilla. One is also a negative (integration of a proprietary application, Pocket, into the browser; it may be included in your “constant erosion of trust” point), but the others are more positive.

                                                                                                                                                                              Mozilla is the organization that let Rust emerge. I’m not a Rust programmer myself but I think it’s clear that the language is having a huge impact on the programming ecosystem, and I think that overall this impact is very positive (due to some new features of its own, popularizing some great features from other languages, and a rather impressive approach to building a vibrant community). Yes, Mozilla is also the organization that let go of all their Rust people, and I think it was a completely stupid idea (Rust is making it big, and they could be at the center of it), but somehow they managed to wait until the project was mature enough to make this stupid decision, and the project is doing okay. (Compare to many exciting technologies that were completely destroyed by being shut out too early.) So I think that the balance is very positive: they grew an extremely positive technology, and then they messed up in a not-as-harmful-as-it-could-be way.

                                                                                                                                                                              Also, I suspect that Mozilla is doing a lot of good work participating to the web standards ecosystem. This is mostly a guess as I’m not part of this community myself, so it could have changed in the last decade and I wouldn’t know. But this stuff matters a lot to everyone, we need to have technical people from several browsers actively participating, it’s a lot of work, and (despite the erosion of trust you mentioned) I still trust the Mozilla standard engineers to defend the web better than Google (surveillance incentives) or Apple (locking-down-stuff incentives). (Defend, in the sense that I suspect I like their values and their view of the web, and I guess that sometimes this makes a difference during standardization discussion.) Unfortunately this part of Mozilla’s work gets weaker as their market share shrinks.

                                                                                                                                                                              1. 3

                                                                                                                                                                                Agreed. I consider Rust a positive thing in general (though some of the behavioral community issues there seem to clearly originate from the Mozilla org), but it’s a one-off – an unexpected, pleasant surprise that Rust didn’t end in the premature death-spiral that Mozilla projects usually end up in.

                                                                                                                                                                                Negative things I remember most are Persona, FirefoxOS and the VPN scam they are currently running.

                                                                                                                                                                                1. 4

                                                                                                                                                                                  I consider Rust a positive thing in general (though some of the behavioral community issues there seem to clearly originate from the Mozilla org), but it’s a one-off

                                                                                                                                                                                  Hard disagree there. Pernosco is a revolution in debugging technology (a much, much bigger revolution than what Rust is to programming languages) and wouldn’t exist without Mozilla spending engineering resources on RR. I don’t know much about TTS/STT but the Deepspeech work Mozilla has done also worked quite nicely and seemed to make quite an impact in the field. I think I also recall them having some involvement in building a formally-proven crypto stack? Not sure about this one though.

                                                                                                                                                                                  Mozilla has built quite a lot of very popular and impressive projects.

                                                                                                                                                                                  Negative things I remember most are Persona, FirefoxOS and the VPM scam they are currently running.

                                                                                                                                                                                  None of these make me as angry as the Mister Robot extension debacle they caused a few years ago.

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    To clarify, I didn’t mean it’s a one-off that it was popular, but that it’s a one-off that it didn’t get mismanaged into the ground. Totally agree otherwise.

                                                                                                                                                                                  2. 4

                                                                                                                                                                                    the VPM [sic] scam they are currently running

                                                                                                                                                                                    Where have you found evidence that Mozilla is not delivering what they promise - a VPN in exchange for money?

                                                                                                                                                                                    1. 0

                                                                                                                                                                                      They are trying to use the reputation of their brand to sell a service to a group of “customers” that has no actual need for it and barely an understanding what it does or for which purposes it would be useful.

                                                                                                                                                                                      What they do is pretty much the definition of selling snake oil.

                                                                                                                                                                                      1. 7

                                                                                                                                                                                        I am a Firefox user and I’m interested in their VPN. I have a need for it, too - to prevent my ISP from selling information about me. I know how it works and what it’s useful for. I can’t see how they’re possibly “selling snake oil” unless they’re advertising something that doesn’t work or that they won’t actually deliver…

                                                                                                                                                                                        …which was my original question, which you sidestepped. Your words seem more like an opinion disguised as fact than actual fact.

                                                                                                                                                                              2. 2

                                                                                                                                                                                It’s a tool like a lot of other things. Sure, you can abuse it in many ways, but unless we know how the results are used we can’t tell if it’s a good or bad scenario. A good usage for a heatmap could be for example looking at where people like to click on a menu item and how far should the “expand” button go.

                                                                                                                                                                                As an event counter, they’re not great - they can get that info in better/cheaper ways.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  This is tricky and also do for survey. I often am in a situation where it asks me “What do you have the hardest time with it” or “What prevents you from using language X on your current project?” and when the answer essentially boils down to “I am doing scripting and not systems programming” or something similar, I don’t intend to tell them that they should make a scripting language out of a systems language or vice versa.

                                                                                                                                                                                  And I know these are often taken wrongly, by reading the results and interpretation. There rarely is a “I like it how it is” option or a “Doesn’t need changes” or even “Please don’t change this!”.

                                                                                                                                                                                  I am sure this is true about other topics too, but programming language surveys seem to be a trend so that’s where I often see it.

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    I feel like they’re easily gamed, too. I feel like this happened with Twitter and the “Moments” tab. When they introduced it, it was in the top bar to the right of the “Notifications” tab. Some time after introduction, they swapped the “Notifications” and “Moments” tab, and the meme on Twitter was how the swap broke people’s muscle memory.

                                                                                                                                                                                    I’m sure a heat map would’ve shown that after the swap, the Moments feature suddenly became a lot more popular. What that heat map wouldn’t show was user intent.

                                                                                                                                                                                    1. 1

                                                                                                                                                                                      from what I understand, the idea behind heat maps is not to decide about which feature to kill, but to measure what should be visible by default. The more stuff you add to the screen, the more cluttered and noisy the browser becomes. Heat maps help Mozilla decide if a feature should be moved from the main visible UX to some overflowing menu.

                                                                                                                                                                                      Most things they moved around can be re-arranged by using the customise toolbar feature. In that sense, you do have enough bits to make your browser experience yours to some degree.


                                                                                                                                                                                      The killing of feed icon was not decided with heat maps alone. From what I remember, that feature was seldom used (something they can get from telemetry and heat maps) but also was some legacy bit rot that added friction to maintenance and whatever they wanted to do. Sometimes features that are loved by few are simply in the way of features that will benefit more people, it is sad but it is true for codebases that are as old as Firefox.

                                                                                                                                                                                      Anyway, feed reading is one WebExtension away from any user, and those add-ons usually do a much better job than the original feature ever did.

                                                                                                                                                                                      1. 1

                                                                                                                                                                                        I’m wondering how this whole heatmaps/metrics thing works for people who have customized their UI.

                                                                                                                                                                                        I’d assume that the data gained from e. g. this is useless at best and pollution at worst to Mozilla’s assumption of a perfectly spherical Firefox user.

                                                                                                                                                                                        1. 1

                                                                                                                                                                                          @soc, I expect the browser to know it’s own UI and mark heat maps with context so that clicking on a tab is flagged the same way regardless if tabs are on top or the side. Also, IIRC the majority of Firefox users do not customise their UI. We live in a bubble of devs and power users who do, but that is a small fraction of the user base. Seeing what the larger base is doing is still beneficial.

                                                                                                                                                                                          worst to Mozilla’s assumption of a perfectly spherical Firefox user.

                                                                                                                                                                                          I’m pretty sure they can get meaningful results without assuming everyone is the same ideal user. Heat maps are just a useful way to visualise something, specially when you’re doing a blog post.

                                                                                                                                                                                      2. 1

                                                                                                                                                                                        never understood why so many people left for Chrome,

                                                                                                                                                                                        The speed difference is tangible.

                                                                                                                                                                                        1. 2

                                                                                                                                                                                          I don’t find it that tangible. If I was into speed, I’d be using Safari here which is quite fast. There are lots of different reasons to choose a browser. A lot of people switched to Chrome because of the constant advertising in Google webapps and also because Google has a tendency of breaking compatibility or reducing compatibility and performance with every other browser, thus making Google stuff work better on Chrome.

                                                                                                                                                                                      1. 14

                                                                                                                                                                                        REPLs are nice but they work well only for reasonably isolated code with few dependencies.

                                                                                                                                                                                        Which is exactly the reason to have all your code broken up in this way. Even if you don’t use REPL.

                                                                                                                                                                                        It’s hard to set up a complex object to pass into a function.

                                                                                                                                                                                        Avoid complex objects then. Everything wrapped in bespoke objects with methods that each don’t even need all of the object’s state is one of the most prominent things that’s wrong about OO.

                                                                                                                                                                                        1. 18

                                                                                                                                                                                          “Avoid complex objects” is a good rule of thumb to keep code clean, but if you’re modeling a complex real-world concept, the complexity needs to be reflected in your model somewhere or it will be an incorrect representation.

                                                                                                                                                                                          1. 13

                                                                                                                                                                                            Ah-hah! A friend of mine once put it very nicely when, in a fit of rage caused by trying to refactor a 4,000-line drive, he just yelled fsck it, this DMA controller here is super murky so of course my driver for it is going to be super murky, sillicon don’t care I gotta write clean code.

                                                                                                                                                                                            Not all things admit a “beautiful” model. A set of tools that only works with things that admit a beautiful model makes for great blog posts but lots and lots of frustration when they meet the messy real world of here are the 14 steps you have to perform in order to get a byte off this bus, yes the writes at steps 6, 7 and 10 are at aliased, unaligned memory locations, why do you ask?

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              But isn’t this what abstraction… does? We have an ugly inconsistent machine that doesn’t solve our problem very elegantly (hardware of some sort) so we sandwich in other, abstract machines of higher and higher level of abstraction until the thing we want to express almost falls right out of just asking the question of the highest level?

                                                                                                                                                                                              (And each level abstract machine does mask over a few of the most glaring problems of the machine just underneath it, so it’s low-dependency and trivial to test and reason about.)

                                                                                                                                                                                              1. 2

                                                                                                                                                                                                When the underlying hardware is an ugly, inconsistent machine, the first layer of abstract machines will have to mimic its model, you don’t really have a choice about that. If it consists of 12 non-consecutive memory-mapped registers that can be written independently, but not concurrently, then you’ll probably have 12 pointers and a mutex (at best – sometimes you’ll need 12…) whether you like it or not, because ultimately data will have to be written in memory. If getting a byte off a bus requires you to do fifteen ugly steps, then your code will have to do them.

                                                                                                                                                                                                Now, in addition to that, you can sometimes wrap this is a unified, ellegant interface. That won’t save the first layer of code but it will at least allow third parties to use it without having to deal with its peculiarities.

                                                                                                                                                                                                If you poke around Linux’ driver tree (I’m mentioning it just because it’s probably the one that it’s easiest to find docs about, it’s not necessarily my favourite), you’ll find plenty of cases where the driver authors had to go to incredible lengths, and sometimes do a bunch of horrendous hacks, in order to get the driver to conform to the unified interface while still keeping the hardware happy.

                                                                                                                                                                                                But even that doesn’t always work. Abstractions inevitably leak, for example – you can’t augment the “unified” interface to account for every hardware quirk in existence, otherwise you end up with an interface so large that nobody implements it correctly. That’s true outside the world of device drivers, too, I’ve seen plenty of codebases where people really wanted ellegant, unified interfaces, and they ended up with 120-function interfaces to “middleware” interfaces that plugged to the real interfaces of the underlying implementation. Most programmers, including yours truly, just aren’t smart enough to navigate the innards of that kind of a behemoth, which is why it looks great in the docs but it’s a bug-ridden piece of junk if you try to run it.

                                                                                                                                                                                                Speaking strictly for device drivers, you also can’t pile up abstractions forever, at some point you have to stop at the unified API everyone expects. E.g. if you pick ten devices from the same family, you could probably come up with a higher-level, more ellegant abstraction than the one their device drivers implement on a given platform, but it would be completely useless because that’s not the interface the next layer in the kernel expects. Sometimes it’s literally impossible to change the next layer (e.g. on Windows, unless you’re Microsoft), but even if it’s technically possible, that’s a whole other can of worms.

                                                                                                                                                                                          2. 3

                                                                                                                                                                                            To clarify, I do still think the described tooling would be useful.

                                                                                                                                                                                          1. 4

                                                                                                                                                                                            I would expect advertising companies to try to abuse this kind of thing for cross-app tracking, more than criminals.

                                                                                                                                                                                            Aww, why are we still making a distinction here?

                                                                                                                                                                                            P.S. Even if I wouldn’t consider advertising company criminal by default, it’d be the correct designation for any that tries to use a covert tracking channel.

                                                                                                                                                                                            1. 6

                                                                                                                                                                                              I love this explanation. Succinct and without trying to rationalize any subjective preferences for “beauty”.