1. 4

    This is going to be less a critic to this article itself, or the metric therein proposed, and more a critic to, let’s say, “approach to truth” of software developers in general. And as a software developer myself, I’m fully included and guilty of much of the same. If I come of as a dick, I apologize, but text allows for a limited range of emotions, and I’m bad at them even in person.

    First, the study quoted has an N of 78, yet calls itself a large scale study. Sure, compared with some of the studies cited in the article itself, with 10 and 18 participants, 78 is big. I couldn’t think of a specific study to compare with, so I just typed “large scale psychology” on Google Scholar and clicked around. In the first page I found this study, with 23533 subjects, and this one, with 1895.

    Now, to be fair, I haven’t actually read these studies, just skimmed the abstracts, but the point is: quantitative studies in software engineering are not even in their infancy, they are so young they can be legally aborted in 67 countries. We all like the validation of “science says so”, but bleeding edge research like is not very trustworthy, and anything it says should be taken with several grains of salt.

    I’ll give to the author that he did use <rant></rant>, so, it’s not like he’s claiming scientific truth, and this sorta compensates for the fact that there’s no link to the study cited in the article (which, when quoting research, that’s kind of a must, IMHO). But I feel that the rest of the article fails to maintain that humility, which is what I’ll try to address next.

    Second, the CCL metric is baked by pretty much nothing. There’s absolutely no other references to it anywhere other than from the author himself. It has not been tested in any empiric studies, which makes it kinda worse than the studies that the article that illustrates the post low key makes fun off, and is not even referenced by other famous people in the field, lacking even the fallacious support of the appeal to authority (broadly speaking. I’m not familiar with the author, but he looks pretty accomplished, so, what he says might have some weight, but it would be more impressive if more people agreed. Martin Fowler doesn’t have scientific evidence for most of what he says, but if you google the terms he uses, he’s not the only one using them).

    Experience counts to a lot in software development. I for sure cannot justify with data and scientific studies 99.99% of the decisions I have made in my career, or of the advices I might have given to less experienced developers. But the mashing together of the (admittedly, illustrative) study, with the authoritative tone of the article, seems very representative of the refusal of developers in general to admit to ourselves that, actually, we’re not really sure of how to write good software.

    Third, and that will be brief because I think it’s better addressed by other comments in this thread: when it comes to actually defining his new metric, the author gives no reasoning for the definitions.

    The very basic definition is already kinda subject: risk of what? But that looks like it could turn into a rabbit hole so I’ll just give it a pass.

    Then there’s the condition:

    It’s scoped first by method/function and then by compilation unit. There’s no other scoping (namespace, class, module, etc.)

    Why? Why is method/function more important than other scopes? Why are arguably wild different types of scope grouped at same level of (un)importance?

    The definition itself doesn’t fare better: what is a symbol? How do you define that cross-languages? What about languages that don’t have exceptions at all? Do we consider only panics, or error codes as well? And then there’s the restriction of “editable code”. There’s some reasoning down the line for it, but it’s just circular: “If you’re just hotlinking to a dll or something, that’s different.” Different how? Why? Is a .jar equivalent to a DLL? What about a library compiled from Cython code? It’s binary, but the source code is pretty much Python. Does it count as editable?

    At the end, my points are the following:

    1. what we do ain’t science. There’s science in trying to understand, and I’m really glad that it’s growing, but the results are not that solid yet for us to take then as gospel.
    2. We need to acknowledge that much of what we hold as true is unsubstantiated, fruit of experience and biased observation. It’s the best we have right now, but we shouldn’t hold to it to tightly, least we find ourselves believing things that don’t correspond to reality anymore.

    PS: I wanna take exception to this specific paragraph, because I feel it misrepresents not only my work, but that of many excellent co-workers I had the pleasure of work with:

    For some odd reason, discussions on frameworks and complexity always devolve into some version of “Dang kids!” vs. “Talentless luddite!” As many people have pointed out, not only do we not teach or talk about incremental complexity, if you don’t have the appropriate buzzwords in your CV, you don’t get hired.

    The bit about buzzwords on CVs certainly rings true, but in my experience complexity is very much discussed in terms of increments, of the price you pay for adding complexity now. We talk about keeping things simple, about “You ain’t gonna need it”, about the dangers and benefits of using 3rd party dependencies and services. It might not become blog posts and tweets, but those concerns have been present at pretty much all discussions in projects I actually worked on. Said concerns haven’t always been heard, mind you, but they have been raised, at least.

    1. 2

      First, the study quoted has an N of 78, yet calls itself a large scale study. Sure, compared with some of the studies cited in the article itself, with 10 and 18 participants, 78 is big. I couldn’t think of a specific study to compare with, so I just typed “large scale psychology” on Google Scholar and clicked around. In the first page I found this study, with 23533 subjects, and this one, with 1895.

      An N of 78 is fairly weak, and an N of 10 is a joke. You can flip 10 fair coins and still have a 0.097% chance of landing all heads (or tails); to put that into perspective, major cloud providers offer higher reliability than the chance of having 10 fair coin flips all land on heads/tails. (Yes, I realize that this is Binomially distributed data not drawn from something like a Student’s T which may be more appropriate for variables under consideration.)

      what we do ain’t science. There’s science in trying to understand, and I’m really glad that it’s growing, but the results are not that solid yet for us to take then as gospel. We need to acknowledge that much of what we hold as true is unsubstantiated, fruit of experience and biased observation. It’s the best we have right now, but we shouldn’t hold to it to tightly, least we find ourselves believing things that don’t correspond to reality anymore.

      S/o to @hwayne for really trying to push this. It’s my biggest pet peeve with technological discussions. A veneer of authority is applied to someone’s opinions and experiences in order to make categorical statements and apply condescension to others’ opinions and experiences. Correctness benefits of FP is the archetypical example here, but the metric discussed in this post is more of the same.

      1. 1

        Yes, I wanna be @hwayne when I grow up XD

    1. 1

      Not to be discouraging or anything, but that looks absolutely horrible. It might be slightly easier to write, but looks terrible to read, plus it invents it’s own, new, never used anywhere, convention for something that already has a convention pretty much set in freaking stone (parenthesis).

      I mean, scratch your itch, by all means, but in terms of language syntax design that just looks like a bad idea all around.

      1. 2

        Not trying to judge, but every convention was once new.

        1. 1

          Fair enough, but being new is not my only criticism, it’s more like the straw that breaks the camel’s back, really.

          1. 1

            Agreed :)

      1. 4

        Hopefully, something. Lockdown + seasonal depression (I didn’t use to live this far north until a year ago) + just regular run of the mill depression have turned me into an absolute couch potato. It’s hard enough to work on the week, and it has been almost impossible to get out of bad on weekends.

        1. 2

          Oof. I did my studies in a fairly cold/dark place, and Seasonal Depression was really tough. Combining that with lockdown must be really tough. Hang in there!

        1. 1

          I mainly use this to push git branches to remote, because I can never be bothered to actually remember (or type) the actual command.

          1. 2

            I’m really happy that OP exists, so I won’t have to carefully read that post about the morality of optimization and write a refutation of it.

              1. 1

                Somehow I missed that. Wow, that’s great.

                I had aliased fuckgit to git push --set-upstream origin master because I miss that one so often.

                1. 1

                  Yeah, that is pretty much the main use case I have for it as well =P

              1. 1

                This is a very interesting post but to be honest the most exciting thing in there was finding out that recurse center is doing it’s thing online now

                1. 1

                  I’m not a wild fan of exceptions in general mainly because of this, although they are way more of a common idiom in Python than in other languages.

                  That said, I like your post, there’s little I disagree with, but the code on your is_invalid_float example almost made me stop reading. The if version has unnecessary branches and returns, could be just return np.isnan(nr) or np.isinf(nr). I wonder if you would feel as compelled to simplify this version. The exception version then goes on to use a naked except, ensuring that it will gobble up any other error that might arise, including things like SystemExit or KeyboardInterrupt. Since you want a safe float number, perhaps OverflowError would be the right exception to catch, or even ArithmeticError, to check for any kind of, well, arithmetic error. When in doubt, I always consult the documentation

                  <extreme nitpicking mode off>

                  1. 3

                    Here’s my problem with this kind of post (yes, it’s a kind: I think I’ve lost count of how many iterations I’ve seem by now):

                    1. It’s a generalization over personal experiences of other people. I.e., the author is making comments about behavior he has seem in other people, which has two issues:
                      1. We’re usually way less forgiving to other people than to ourselves. When we look up something on Stack Overflow, it’s an efficient use of resources, we’re not gonna reinvent the wheel here, we have business problems to solve! When other people do, they’re lazy and not thinking hard enough about the problem at hand.
                      2. Last occurrence bias (this is probably not the right name but I can’t be bothered to look it up): The author probably feels, likely, strongly, that he’s seem this kind of behavior in others, and a lot. I bet, though, that if they try to actually write down the actual instances of said behavior, it will be a somewhat underwhelming number.
                    2. This is advice that will probably miss whoever needs it, and instead hit (and hurt) the people who least need to hear that they shouldn’t look things up: beginners, specially beginners of underrepresented minorities. People that copy and paste mindlessly from Stack Overflow hardly will stop to read a long post and they think about it, have some introspection, and realize they need to change their ways. Beginners, though, being already insecure, might take away the wrong thing and think that they’re even more useless than they already feel.

                    Some, in short, this kind of post is likely an exaggeration over personal experience, and will hurt more than help people who read it.

                    “Well, that’s nice, but I actually saw a person doing this the other day, so what do you suggest? Should I just let them ruin the codebase with crap copy and paste from SO?” Glad you asked. Coach them. Directly. Act on the actual instance of the issue. Maybe even send them this post? At least it might do some good, then. But ranting about it online will pretty much only miss the target and hit “innocents”.

                    Disclaimer: I might very well be incurring in 1.1 and 1.2 myself, so, take this with a grain of salt.

                    1. 2

                      It’s a generalization over personal experiences of other people. I.e., the author is making comments about behavior he has seem in other people

                      Actually, it’s a generalization over my own personal experience. I don’t work in a tech company and I have very few friends that are into programming, so I don’t know the behaviour of other people in this regard. In fact, nearly all the post is written in first person!

                      People that copy and paste mindlessly from Stack Overflow hardly will stop to read a long post and they think about it, have some introspection, and realize they need to change their ways.

                      This could be true.

                      Beginners, though, being already insecure, might take away the wrong thing and think that they’re even more useless than they already feel.

                      Hopefully beginner programmers can read an article and understand its meaning! I think I emphasized enough the idea that Stack Overflow should be used, it just shouldn’t be used mindlessly.

                      All in all I believe that it is true that the article is pretty generic, but it’s the way I wanted it. I also believe that it can either be useless or useful, depending on who reads it, but not harmful.

                      PS: I’m glad you liked the theme of my blog (at least :P). It is a Jekyll theme and I made it available on GitHub. Here’s the link if you want to check it out.

                      1. 1

                        PS: Loved the theme on your blog though, might steal it =)

                      1. 2

                        Brazilian developers will have a hard time using this tool.

                        1. 2

                          Why?

                          1. 3

                            I’m not sure this is the reason @tonnydourado made that remark, but “cu” apparently means “ass” in Portuguese.

                            Oh, and of course, “git” is slang for an unpleasant individual…

                            1. 2

                              Yeah, pretty much, it’s the “cu” part.

                              1. 1

                                Oh dear 😅

                                1. 2

                                  Maybe they can apply this just for some clients :)

                          1. 3

                            I like syncthing, I have used it before, but I’m on an iPhone now and there’s no client for it, so, too bad.

                            But what I really wanna say is that I’m glad this is not yet another masturbatory nostalgic rant about how things where great when computers where just shitty terminals running BASIC, or shitty CPUs running DOS, and how the web ruined everything and javascript and npm are the literal antichrist.

                            1. 1

                              I also looked for an iOS client for Syncthing, and didn’t find one. I had a look at the protocol specification, and it looked like it should be quite doable to implement it as a Files-compatible thing. If only I had unlimited time to work on side projects…

                            1. 5

                              No, defects are the fault of individual humans and small groups of humans. Defects caused by combining two things that work fine in isolation but not together is the fault of the programmer that integrated them. 95% of the time it’s because they made assumptions along the lines of ‘assume postcondition A implies precondition X’ or just ignored the guarantees and requirements of the two systems.

                              More generally, stuff like this…

                              Just scapegoat the human and let the broader system get off scot-free. Don’t ask if the equipment was getting out of date, or if the training program is underfunded, or if the official processes were disconnected from the groundlevel reality. When all defects are the fault of the human, then we don’t have any tools to address defects besides blaming humans. And that’s completely inadequate for our purposes.

                              …ignores that systems are set up by and signed off by humans. None of this stuff just spontaneously arises. Depending on random npm packages with no security guarantees or code review isn’t a system problem, it’s the fault with the person that added that dependency or the person that signed off on allowing random unreviewed dependencies.

                              Making things “system problems” is just a way to shift blame from individuals to everyone because “you can’t fire EVERYONE”. But actually the people that sign off on these systems, the people that get paid HUGE amounts of money on the basis that they’re supposed to be held responsible when something goes wrong, but who somehow never are? The blame is on them. If you tell developers not to review the contents of NPM packages because you have a tight schedule and don’t have time for it then YOU AND ONLY YOU are responsible for the consequences of that decision. Not the system. Not the collective.

                              If you don’t have time or resources to do your job professionally then you need to take a stand and tell you manager so. In writing. “I can’t be sure of the quality of this if you do not give me time to either review the dependencies deeply or write that code myself”. Be a professional.

                              The only way the public is going to get software that works and is safe and isn’t riddled with security holes is if programmers start creating an expectation of professionalism. Nobody says ‘oh well this bridge doesn’t REALLY matter so we can just cut corners’. Software might not always have the same potentially bad consequences for physical safety (although often, increasingly often, does) but has serious consequences for emotional, psychological and privacy safety. What’s more people should be able to expect software to work, to always work, to work properly and accurately and safely. If that costs more then it costs more. I’m sure unsafe bridges would be cheap too.

                              1. 20

                                Having, uh, worked on, well, not bridges before, but some physical objects, I really, really hate it when reasoning by analogy is brought up, in the form of bridges or cars or whatever.

                                First of all, everybody asks which corners can be cut, all the time. An unsettling amount of regulation is in place precisely in order to prevent people from cutting the wrong corners, and oftentimes precisely because someone did cut the wrong corner in the hope of getting slightly richer, but they ended up killing someone in the process. (Yay for insurance, though – they really did get slightly richer most of the time).

                                Second, in other engineering disciplines (electrical, in my case, although I’ve always been an intruder in that field and haven’t really done any EE work in years now) the fact that “the system” sometimes doesn’t work is perfectly understood. Yes, it’s usually one person (or a group of persons) that makes the mistake – the efficient cause of the mistake, if you will – but in a well-lead team, that rarely happens in isolation from others, and purely as a personal failure. Most bad decisions I’ve seen happen at the intersection of:

                                • Incomplete understanding of a problem’s implications among management ranks
                                • Incomplete understanding of requirements, deployment constraints, usage patterns etc. among engineers
                                • Lack of patience among management ranks (“I don’t need all these details, just give me, like, an executive summary” – it’s not like a complex problem magically becomes simple once the gremlins realize you’re busy)
                                • Impedance mismatch at the middle-management layer

                                In these cases, many mistakes really do end up being “the system’s” fault – in that, although someone’s wrong calculations or wrong prioritization really did end up causing the screw-up, no one, not even that person, can really be blamed for what they did.

                                “Defensive” responses to this are absolutely common (and this article is one of them, too), at both ends of the corporate ranks. Upper management & co. insists that personal responsibility is key and that if you think something is wrong, you should speak up and improve the process – gleefully ignoring the fact that doing so usually gets you ignored (see points 1 and 3 above), and sometimes fired if you go over middle management’s head. Middle management and engineers will insist that fault rests solely with the stewards of the system – gleefully ignoring the fact that, indeed, no system is completely impervious to someone doing their jobs sloppily.

                                Of course, the people at the bottom of the hierarchy don’t have any subordinates to shift the blame to, which is why the former reaction ends up getting more street cred, but that’s a whole other story…

                                1. 5

                                  Ok, it’s the fault of individual developers. What then? What are you offering as a solution to that from a practical point of view? Empirically, appeals for professionalism and accountability clearly don’t work.

                                  Besides, bridges also seem to collapse with some regularity in places like the US and Italy, so I don’t know if that’s a useful counterexample and I don’t know if appealing to professionalism is going to solve that problem either.

                                  1. 4

                                    Depending on random npm packages with no security guarantees or code review isn’t a system problem, it’s the fault with the person that added that dependency or the person that signed off on allowing random unreviewed dependencies.

                                    Did you … did you read the thing in the link?

                                    1. 2

                                      programmers start creating an expectation of professionalism

                                      I observe a power imbalance here. Most programmers I know are very quick to shift the responsibility away: “It is not my job to prioritize” and “I need to ask my manager”. On the other hand, managers practically never do that: “I will clarify that” or “I will follow up on that”. It does not mean that managers actually solve more issues than programmers but it creates a perception that the managers are the doers and programmers are the whiners. If programmers never take the responsibility, they will never get the power either.

                                      Disclaimer: Of course, there comes a risk with taking responsibility. In some circumstances it might even be a trap.

                                      1. 1

                                        This is both a completely true and also effectively useless. As a purely practical consideration @hwayne’s approach to analysis and addressing issues has a higher likelihood of success than blaming human error.

                                      1. 2

                                        Looks nice, would be cool if it was more mobile friendly, though? I quick tested with the responsive design mode on Firefox, it just gets smaller, which in an actual device is kind of annoying.

                                        1. 5

                                          I adjusted index.html the grid to be responsive.

                                          1. 7

                                            That’s quicker support than services I paid for, hahahah =P

                                        1. 3

                                          I really like the thrust of this idea, but I’m virtually certain this grid is an abuse of the <aside> tag. Surely there’s a way to achieve the same thing with semantically correct HTML?

                                          1. 6

                                            Also:

                                            • acronym is deprecated.

                                            • your index.html lacks its opening html tag.

                                            • <h3>Ordered lists</h3> should be an h4.

                                            • <label>Number:</label> should be an h4, and the subsequent input elements should be labelled:

                                              <h4>Number</h4>
                                              <label style="display:block"> <!-- ideally avoid <br/> -->
                                                <input type="radio" name="radioSet" value="uno"/>
                                                Uno
                                              </label>
                                              <label> ...
                                              
                                            1. 4

                                              You must be fun at code reviews.

                                              Just kidding, I actually admire people which a keen eye for detail. Carefully examining other’s people code can be as helpful if done with kindness as it can be bad if done condescendingly or arrogantly, and you didn’t really do it in either of the later bad ways =)

                                              1. 3

                                                Thanks! I will fix that.

                                              2. 1

                                                The project is build around the premise of default CSS starter kit. Instead of using CSS reset you use something like this to set the default style. You will probably need classes to make more complex grid. The grid logic is taken from: https://github.com/vladocar/infinity-css-grid .

                                              1. 6

                                                I don’t like to blame Java for all woes in programming, but it’s really hard not to blame this one on it. Verbs should generally not be entities. The naming gives you the clue: if you’re having to invent nouns from verbs to name your classes, then perhaps they should not really be classes.

                                                Now, in less restrictive languages, this is a simple fix: create a function instead. But then Java came along, decided that everything should be a class (which is actually the problem. Everything being an object is fine, and actually has nice benefits, like easily enabling functions as parameters, as long as you can have function objects =P), and poisoned the minds of so many young people.

                                                1. 1

                                                  Did anyone else read “Bad motherfuckers: how to recognize and avoid them”?

                                                  1. 2

                                                    Just check their wallet.

                                                  1. 3

                                                    I do it in Python occasionally.

                                                    1. 1

                                                      Python only allow it to a certain extent. You can monkeypatch any module, even stdlib ones, but you can’t monkeypatch the builtin types or instances of builtin types. modules from C extensions are not monkeypatcheable as well, I think?

                                                      1. 1

                                                        Have been bitten by this in python when a library I imported redefined a global object. It was not a fun debugging session.

                                                        1. 2

                                                          Yeah, can imagine. I usually only use it under extreme circumstances, and with a context manager, to minimize the impact. Kind of similar to what patchy does (although patchy is even more extreme and less fun to debug)

                                                      1. 13

                                                        The below essay is Frankfurtian bullshit. I wrote it in the same style as https://shouldiusethreads.com/, because I believe that it is also bullshit of the same type; all of the statements are true, or at least very very hard to falsify, but it’s written primarily to offend, not based on its actual truth or falsehood.


                                                        Should I use the filesystem?

                                                        No

                                                        Some people would have you believe that NULL is the worst mistake in computer science. They make a good case for it, but they’re dead wrong. Shared mutable state is the worst mistake in the history of computers. Ask anyone who has debugged both a segfault and a race condition. They will assure you that the latter is 10-100× more difficult to solve.

                                                        Look at a list of files that your system has touched recently. How many of them have actually been manipulated by more than one program? I’m mostly thinking of the hidden ones, like .bash_history and the .dvdcss directory. Outside of highly-visible but comparatively-rare circumstances, the filesystem is an excessively general abstraction for the “silent majority” of its use. The poorly-defined concept of filesystem atomicity result in a system that simultaneously has both excessive synchronization and not enough synchronization for the purposes of the same application (PostgreSQL, in the case of those two links). It’s impossible to design an fsync() call that provides full-filesystem atomicity while preventing two applications on the same computer from blocking each other, because fsync() is fundamentally a filesystem-global lock.

                                                        What’s worse is that the default behaviour is to allow limitless race conditions. A process has access to the same filesystem space as other processes in your machine, and can do whatever they want with it (as long as they’re running as the same user, which is the default behaviour when you launch processes). You have to take extreme pains to avoid accidentally doing the wrong thing. You’ll probably mess it up, and the symptoms will show up in an unrelated part of the multiprocess system 10 minutes later. There is no shame in making such mistakes — we’re only human, after all. The shame is in believing ourselves super-human, and reaching for a tool that we don’t need, in full knowledge that we’re likely to shoot ourselves in the foot with it. Nine out of ten times, this tool is the filesystem. If you’re asked to dig a hole and you point a pistol between your feet to get the job done… you should have used a shovel.

                                                        But, perhaps your problem requires multiple processes to operate on the same data. In that case — you still shouldn’t use the filesystem! Consider using message passing instead. A good model is an overseer program, which organizes the work to be done and aggregates the results, spawning worker programs to run the actual computations. This is not only much more robust, but it scales better, since you could distribute these programs across multiple computers later on*. This approach is more useful, too, because often the user of the program may have a more novel idea about how to distribute work, and if they can invoke the worker processes themselves, they’re empowered to try it out. This is easier to test and debug for the same reasons. And, in addition to all of these benefits, you get brand new virtual memory scratch space which is much more difficult to fuck up.

                                                        What’s worse is that, while there has been a great deal of attention paid to memory-safety, filesystem-safety in programming languages is essentially nonexistent**. Python, Java, Rust, even Haskell expose almost identical APIs for manipulating files, and that interface is terrible. Database designers since before I was born have known that user data and query code should be connected using prepared statements only, but that’s not how you manipulate paths on the filesystem on any of the major operating systems. If you want to connect user input that might potentially contain a forward slash, the best thing that language abstractions might provide is a function that will error out in the presence of “forbidden characters.” At least HTML has standardized escape sequences; how is it acceptable that, in 2020, we can’t have forward slashes in file names? You want to scrape the IMDB and use it to populate file names? Sorry; those episode titles have forward slashes in them, so you’re going to have to invent some arbitrary, non-standardized character substitution and hope it doesn’t result in a naming conflict. “Don’t fuck with user input” is a pretty low standard for API design, and the filesystem paths fail it.

                                                        Don’t use the filesystem! It’s a trap!

                                                        * NFS is not a solution. Because it’s designed to be transparent to the application, it can’t be both reliable and fast, because the application doesn’t give it enough information to know whether an operation should be retried or failed out if something goes wrong.

                                                        ** The closest you can get is sqlite, which is great, but it can’t really fix the pessimistic synchronization, and it doesn’t work very well with existing tooling like git.

                                                        1. 6

                                                          Honestly, this mostly makes sense to me? The fact that we don’t have any better solutions to the use cases filesystems address doesn’t mean that a global shared filesystem is a good idea to begin with. In fact, if you look at more recent OSs that cared less about retro compatibility, like the mobile OSs, they all have some sort of per-app scope for filesystem, and in iOS specifically it’s something very abstracted over.

                                                          I mean, if you were designing a OS today, without any care for any retro compatibility, without any need to support any sort of posix API, would you really provide files and a filesystem as an abstraction for storage? I don’t think I would.

                                                          1. 1

                                                            I can’t decide.

                                                            On the one hand it feels pretty archaic to just store labeled blobs of bits vaguely sorted in a tree and maybe with an extension as a rough type indicator. How are we going to explain this to the kids swiping on the TV? ;) And that in times where we have cloud everything and our phones have share buttons to avoid dealing with files! I am totally on-board with this accidentally true rant.

                                                            On the other hand pretty much every order I need I can create with the wonderful filesystems we have and I prefer that to dealing with having my files scattered over sharepoints and onedrives and google docs and whatnots.

                                                            1. 2

                                                              How are we going to explain this to the kids swiping on the TV?

                                                              The same way it was explained to me? They can learn like anyone else, it’s not unlearnable.

                                                              1. 2

                                                                Your second alternative is not the only other option to filesystems though. It might seem like the only feasible one, but I’m not constraining myself with such minor concerns as “is this even remotely viable in the real world at all”.

                                                                What if we could have one (or a couple) rich, programmable, safe, and portable interface that every program could (and did) use? What if you could have even more flexibility (or rigidity) at your fingertips to organize your data, in pretty much any device?

                                                            2. 1

                                                              You nailed it! I would just add a few solutions like “use INT 13h directly” or “print your data on paper”.

                                                            1. 1

                                                              This seems all very nice indeed, but every time I embark on some customization journey I eventually give up, for one reason: reproducibility.

                                                              Do you use more than one computer? Now you have to replicate all of that in all of them. Do you use more than one OS? Same. Reinstall/update the OS? Either do everything again or have to fix it because the update broke it. Need to do something in someone else’s computer? It’s gonna be a pain because your muscle memory will be all over the place.

                                                              None of this is unsolvable, or even arguably that hard. But I don’t usually have the patience to do it, so, I tend to stick to vanilla as much as possible. If something is really annoying I might change a config or add a program to help (clipboard manager is a thing I can’t live without, for instance).

                                                              1. -2

                                                                I appreciate the time that you took to write this post. I know that the Python community, including myself, can be aggressive and unpleasant in online discussions.

                                                                The PyPy project, for example, is currently dependent on a Python 2 interpreter to bootstrap and so will be maintaining their own either for as long as PyPy exists, or for as long as it takes to migrate to bootstrapping on Python 3 (which they seem to think is either not feasible, or not something they want to do).

                                                                This quote is emblematic of the contempt that you display towards Python users. You imply with your words here that porting RPython to Python 3 is “feasible”, but that ignores the cost of labor. Multiple projects have found that Python 3 porting is too expensive to tackle without dedicated funding and support, including not just Django and PyPy, but also Twisted and entire distros like Ubuntu and Gentoo; if those projects cannot find funding then they do not do the porting.

                                                                I don’t understand your denigration of interpreters. Producing and maintaining an interpreter is non-trivial, a language community should strive to have a heterogeneous collection of interpreters and compilers besides the reference implementation, and the big list of Python 3 libraries pledging to drop Python 2.7 is missing not just PyPy, but also Jython and IronPython. While some implementations, like MicroPython have switched exclusively to Python 3, others have not.

                                                                To paraphrase somebody else’s insight from an IRC discussion, perhaps, as with Perl and Raku, it is the case that Python’s community became too ideologically burdened, and the fork is the result of an ideology operating without constraints. After all, my observations are only weighty because of the inertia of code running on computers, and perhaps that is the only argument in favor of Python 2: That it is the way that things are being done, regardless of the loudness of the claims of Python 3 proponents. I certainly am not here to praise Python 2 either, but to bury it; in fact, were the language still in one piece, I would only have to dig one tomb and write one eulogy. I need to credit glyph for being the one to first paraphrase Marc Antony, over a decade ago at PyCon; I was so slow to understand what everybody else was trying to point out.

                                                                Though there is one thing I think gets overlooked a lot: usually, the anti-Python-3 argument is presented as the desire of a particular company, or project, or person, to stand still and buck the trend of the world to be ever-changing.

                                                                But really they’re asking for the inverse of that. Rather than being a fixed point in a constantly-changing world, what they really seem to want is to be the only ones still moving in a world that has become static around them.

                                                                I am not asking for either of these positions. First, let me dispel this idea as you’ve worded it. I understand that a language is not fixed in stone, and that it will constantly change and evolve with usage. Indeed, I believe that languages are expressive media, and that the main purpose of language is to send messages and communicate, but since our understanding of the world changes over time, our languages must change as well. (Also, explicitly, I am not wearing any corporate hats. I have many commit bits, but the only relevant plan I currently have is to remove Python entirely from all active codebases, incrementally and immediately.)

                                                                My argument, in ten words or less, is: Okay, yes, but must we have changed “2” to “3”? The massive amount of backporting done so far has revealed that no, most of the new features in Python 3 could have been put into Python 2. People had already, as you pointed out, survived having to change with to be reserved and then a keyword, and that change was far more intrusive than anything in the 2-to-3 changelist. Meanwhile, the modules that ought to have been killed off directly, most notably asyncore, were left in Python 3 despite being effectively unmaintained and long since rotting.

                                                                1. 17

                                                                  the only relevant plan I currently have is to remove Python entirely from all active codebases, incrementally and immediately

                                                                  Surely it would be less work to migrate your Python code to Python 3 than to rewrite it in an entirely different language.

                                                                  1. 12

                                                                    He’s having a tantrum.

                                                                    And I can say that, having overseen the 2 to 3 conversion for a large adtech codebase and maintained 2/3 compatible libraries. I did that because it was both challenging and required really creative, often perverse, solutions. And this was with my mental model very carefully keeping track of string/binary boundaries. It wasn’t easy and people who balanced both interpreters and didn’t collapse under the devilry of duplicate code (and subsequent bug duplication) have my admiration.

                                                                    If I could move a stodgy adtech company to 3 for active projects, then he can as well.

                                                                    I’m not a super coder, just someone who adds asserts and routinely retests her assumptions. If you’re not on 3, it’s either a big ball of mud that should’ve been incrementally rewritten/rearchitected (thus exposing bad design) or you expected an ecosystem to stay in stasis forever. If you’re the latter, compile from source and lockdown your environments more than ever cause you’re on your own.

                                                                    Already I’ve witnessed removal of python 2 from distributions! It’s getting killed regardless of how anyone feels. Digging ones heels in is just silly.

                                                                    1. 0

                                                                      You should work on your mind-reading. My complaint is mostly that, for over a decade, PyPy developers have been consistently marginalized by the PSF. This forced deprecation of Python 2 is merely one more brick in the wall. While I am fortunate to be independently wealthy, and I also barely contribute to PyPy, the typical PyPy developer is a hard-working academic who is trying to dramatically speed up the Python ecosystem, and the PSF’s continued snub of their work is disappointing, especially given their continued endorsement of corporate-sponsored JITs hacked on top of CPython, like Unladen Swallow, Pyston, and Pyjion.

                                                                      I’ve overseen 2-to-3 conversions as well. I’ve seen companies that can do it fine, companies that are ready with 2to3, and companies that are totally hopeless. I’ve seen companies that are so behind on security patches that they cannot upgrade away from Python 2, and companies that greenfield with Python 3. It is hard but possible. The main sticking point, which folks seem to constantly paper over, is that Python 2 and Python 3 are fundamentally different langauges, with different grammar and semantics. Indeed, PSF representatives have explicitly asked me to not talk about Python 2 and Python 3 as if they are different, and when I’ve asked followup questions, they’ve indicated that it is because treating the two languages as different undermines the porting effort. This is how the system works.

                                                                      It is funny that you mention “compile from source;” I use Nix, and so of course there will be CPython and PyPy for Python 2.7 available for a very long time via nixpkgs. Again, I am independently configured, and I am not a part of any core development team. But I am not “on my own;” I am one of a community which maintains the ports tree.

                                                                      1. 3

                                                                        Python 2 and Python 3 are fundamentally different langauges, with different grammar and semantics

                                                                        No they’re not. The existence of 2/3 compatible packages utterly shits on your point.

                                                                        The only “breaking” change that you seem to be whining about as soooo different is the splitting of a primitive type. That’s it. Splitting a fucking type into two. That’s not a fundamental language difference and you know it. You’ve made backwards incompatible changes in your code bases and most likely have split classes and refactored. Tell me, was your program utterly grammatically different? Probably not.

                                                                        You didn’t whine about long/int merging because it’s a trivial alias to add.

                                                                        But splitting a type is always a lot more contentious and difficult (probably verging on the biggest type of breaking change possible).

                                                                        Removing “print” so that it could be substituted with a function hardly counts as a massive grammer change. And the addition of coroutines is strictly an “add on”.

                                                                        I’m not going to even bother with your “mother loved you best” vis a vis PyPy. Jython has been effectively dead for years (unless they finally got invokedynamic support to actually get usable speeds - I last checked two years ago, so sue me).

                                                                        I don’t know about the others as they’ve been more like toys than have capability of supporting the ecosystem as-is. You can’t just demand equal buy in without supporting the ecosystem as-is - PyPy literally had to make c-extensions work after claiming for years it was impossible to do (which was absurd - I remember reading one of the JVM Ruby interpreters ended up supporting c-extensions through parsing the c-extension source code into compatible libraries - if ruby could do it, so can python).

                                                                        1. 6

                                                                          Please lower the temperature on your posts. You don’t need to curse and make insulting characterizations of his post to make your points.

                                                                          1. 11

                                                                            Your points withstanding (I should be less overtly crude), the GP’s routine use of “contempt” as an accusative is nothing short of insulting. He paints himself as the victim of the PSF and anyone who agrees with them as doing their bidding. It is extremely unkind and quite conversation terminating.

                                                                            I get that you feel you’re being fair, but it comes across as uneven enforcement amidst the unfair characterizations and portrayal of an anti-Python 2 conspiracy, when in reality it’s an interpreter hide-bound by Guido’s rules to keep it as simple as possible (thus hamstringing backwards compatibility like seen in the JVM), a hold over from the early anemic days of Python and something that’ll never change.

                                                                          2. 2

                                                                            The existence of polyglot libraries does not “shit[] on my point;” it merely shows how far people can be convinced to go in pursuit of a satisfying end-user experience. The existence of libraries which compile under both a C compiler and also a C++, Objective-C, Zig, or etc. compiler does not invalidate the point that the languages are different, regardless of whether the program texts are identical in those cases.

                                                                            Here are two short interactions with two CPython versions. I contend that they implement distinct grammars and semantics from each other.

                                                                            >>> {1:2,3:4}.items()
                                                                            [(1, 2), (3, 4)]
                                                                            >>> {1:2,3:4}.iteritems()
                                                                            <dictionary-itemiterator object at 0x7f704a55f170>
                                                                            >>> nonlocal = 42
                                                                            >>> class Foo: pass
                                                                            ...
                                                                            >>> Foo.__mro__
                                                                            Traceback (most recent call last):
                                                                              File "<stdin>", line 1, in <module>
                                                                            AttributeError: class Foo has no attribute '__mro__'
                                                                            

                                                                            And the other one.

                                                                            >>> {1:2,3:4}.items()
                                                                            dict_items([(1, 2), (3, 4)])
                                                                            >>> {1:2,3:4}.iteritems()
                                                                            Traceback (most recent call last):
                                                                              File "<stdin>", line 1, in <module>
                                                                            AttributeError: 'dict' object has no attribute 'iteritems'
                                                                            >>> nonlocal = 42
                                                                              File "<stdin>", line 1
                                                                                nonlocal = 42
                                                                                         ^
                                                                            SyntaxError: invalid syntax
                                                                            >>> class Foo: pass
                                                                            ...
                                                                            >>> Foo.__mro__
                                                                            (<class '__main__.Foo'>, <class 'object'>)
                                                                            

                                                                            I am not sure why you characterize my points as “whining” and then agree with them by substantiating the needful evidence. It seems that you are angry with me because I have points, not because I don’t have points.

                                                                            By using a broad brush to label various Python projects as “toys” or “dead”, you only emphasize the culture of contempt which I highlighted in my thread-starting comment. Moreover, you’ve said the quiet part out loud: The reason for this contemptuousness is a tribal belief in the power of C extensions over plain old Python code. What’s remarkable to me is that this belief isn’t actually in the service of speed or compatibility; the PyPy project produced not just PyPy, but also CFFI, in order to put those claims to rest. Instead, the belief seems to be simply one of majority rule: Because CPython is the popular standard, CPython must remain the popular standard. To quote from The Power and The Glory:

                                                                            And though the hard times are really due to me, it is still in me to quell all this unrest

                                                                            Things must stay, there must be no change; anyway, time to rearrange

                                                                            You must believe me, that there’s been no such betrayal, all that I’ve done, I’ve done for you

                                                                            Things must stay, there must be no change; anyway, time to rearrange

                                                                            There must be a reason why plans have turned around! Not only in me, it must be in you too

                                                                            Things must stay, there must be no change; anyway, time to rearrange

                                                                            1. 3

                                                                              Uh, no. With the exception of nonlocal as a keyword (which btw is no different when with as a keyword was introduced), all the examples cited are properties on data objects, which is categorically not “grammar”.

                                                                              Grammar pertains to parsing the language into an syntactically valid AST from which a bytecode suitable for the VM can run on. The only incompatible grammatical changes I can think of off the top of my head are:

                                                                              • with (python 2.4)
                                                                              • print to print() (Python 3)
                                                                              • nonlocal (Python 3) (similar to global except restricted to scopes excluding global)

                                                                              That’s it.

                                                                              Write a proxy for the builtins and you can support older method returns without a problem. That’s why six was did it’s job so well.

                                                                              By using a broad brush to label various Python projects as “toys” or “dead”

                                                                              Does it support the ecosystem? If not, then not very comparable with CPython in now, isn’t it?

                                                                              As to dead, Jython still doesn’t do invokedynamic and that’s pretty critical to decent performance. Python is well known to be pretty slow. Jython is even slower. That heavily constrains use case to “we have to run a Python script on the JVM” and that’s about it. And it’s also stuck on 2. So it’s left behind in time to a shrinking ecosystem of pure python 2 modules. Talk about useful 🙄

                                                                              Moreover, you’ve said the quiet part out loud: The reason for this contemptuousness is a tribal belief in the power of C extensions over plain old Python code

                                                                              Actually, I didn’t. You made that up all by yourself honey.

                                                                              I only cited the support of C-Extensions as part of recognizing the audience demand for being able to use PyPy in a broader case than just pure Python and rare/underused one shot ports of C libraries and C-compatible library exports (did several for previous employers) that typically used C-extensions/Cython.

                                                                              You’re so wrapped up in inventing enemies that heap contempt on you, but it’s just fellow engineers raising their eyebrows at someone being overly dramatic. Lol contempt. 😂😂😂

                                                                              1. 12

                                                                                I need to go update it for 3.8 final and things coming in 3.9, but I maintain a list of ways to “break” Python, geared at library authors who want to trigger syntax or import errors as quickly as possible when someone tries to use one of their libraries on an unsupported Python version.

                                                                                It also serves as a sort of tour of changes to Python syntax over the past decade.

                                                                                1. 2

                                                                                  I like this! It shows the grammatical additions (SyntaxError raising ones) fairly well!

                                                                      2. 4

                                                                        I don’t have any real opinion on python2/3 myself. However, if the language you use forces you to make changes which you disagree with, then it is not unreasonable to at least consider the other options out there. Maybe it’s simple enough for them to port from python 2 to 3, but will they then have to port from 3 to 4 in the future? Maybe porting from 2 to 3 isn’t a massive deal, but they weren’t super happy with python for other reasons, and it was the straw that broke the camel’s back.

                                                                        1. 7

                                                                          It is more or less twenty years since discussion started about possible breaking changes that eventually became Python3 and 14 years since “official” start. We are not even at the first step for Python4. Based on how often most people change jobs and projects in this industry, it is very unlikely they will need to port to 4 or even worry about what that will entail for a completely imaginary version of the language. They are likely to go through far more impactful transitions before then.

                                                                          These days I mostly write Python 3 and on occasion still Python 2. I find the notion that they are different languages absurd.

                                                                        2. 2

                                                                          Yes, and it would also be less work to use Python 3 than to write an entirely new language. However, we have reasons, and so we produced Monte, a modern flavor of E. We were already leaving Python, and we were already rewriting our Python-based virtual machine in Monte.

                                                                        3. 27

                                                                          This quote is emblematic of the contempt that you display towards Python users. You imply with your words here that porting RPython to Python 3 is “feasible”, but that ignores the cost of labor. Multiple projects have found that Python 3 porting is too expensive to tackle without dedicated funding and support, including not just Django and PyPy, but also Twisted and entire distros like Ubuntu and Gentoo; if those projects cannot find funding then they do not do the porting.

                                                                          I’d like to think I know a thing or two about large projects porting, and about funding situations.

                                                                          (for the uninitiated: I got my commit bit on Django in 2007, was its release manager from then up to about 2014, served multiple terms on the framework’s technical board, and am currently in my fifth year on the board of the DSF, the nonprofit foundation set up to support Django and its community, including by fundraising for, and then funding, its development)

                                                                          Also, hi. I am a Python user, and I’d seriously dispute your assertion that I have “contempt” for Python users.

                                                                          I don’t understand your denigration of interpreters. Producing and maintaining an interpreter is non-trivial, a language community should strive to have a heterogeneous collection of interpreters and compilers besides the reference implementation, and the big list of Python 3 libraries pledging to drop Python 2.7 is missing not just PyPy, but also Jython and IronPython. While some implementations, like MicroPython have switched exclusively to Python 3, others have not.

                                                                          I don’t understand why you interpret it as “denigration”. You’ve repeatedly indicated your dislike for the Python core team’s and the PSF’s messaging that Python 2 is no longer supported, and you’ve argued that Python 2 is supported for as long as alternative Python 2 interpreters are supported. My point is that while a supported interpreter is a necessary thing for calling Python 2 “supported”, it is not sufficient. When people say they want to keep using Python 2, they mean they also want to keep using all their favorite packages and libraries, and many of those simply do not run on Python 2 anymore. And as far as I know, of non-CPython implementations still supporting Python-2-the-language, only PyPy has ever been robust enough to support some of the really important packages like NumPy/SciPy.

                                                                          Anyway, this is why I bring up the question of whether you’re willing to commit to also maintaining a complete Python 2 ecosystem including the big third-party packages. The Linux distros who’ll be maintaining Python 2 interpreters for years to come have committed to that, but as far as I can tell nobody else has. And without the ecosystem I don’t think you have a sufficient “Python 2” and as I’ve told you directly and also mentioned in the post, I think you create unnecessary and unwanted burdens on those packages’ maintainers if you insist on saying “Python 2 is still supported”.

                                                                          The massive amount of backporting done so far has revealed that no, most of the new features in Python 3 could have been put into Python 2.

                                                                          OK, but now make a Python 2 that cleanly implements the string change and the reorganization of the builtins and the standard library and doesn’t create a massive backwards-compatibility gap to older Python 2 versions. I don’t think that’s possible, and I do think that at least the string change is an absolutely necessary thing for Python’s future.

                                                                          1. -3

                                                                            The Python Packaging Authority (PyPA) is looking to get rid of Python 2 support, at which point I suppose you’ll shift even more of the burden to the remaining Python 2 users somehow, despite the singular package index still being squarely under PSF control.

                                                                            The rest of your post fails to address my point. If you can’t stop talking past me, then please don’t bother. You can’t even look at a Python 2 user without the insinuation that they must be responsible for the entirety of their ecosystem.

                                                                            Frankly, I think that you show your hand when you say “really important packages like NumPy/SciPy.” That’s the direction that you want Python to go in.

                                                                            1. 16

                                                                              at which point I suppose you’ll shift even more of the burden to the remaining Python 2 users somehow

                                                                              Well, I’m not trying to be mean about this, but: this is what the “entitlement” section of the post was about. The PyPA folks don’t owe you a Python-2-compatible packaging toolchain. The Django team doesn’t owe you a Python-2-compatible version of Django. And so on.

                                                                              If those teams have decided they only want to support Python 3, they have the right to do that! And if you want to keep using those tools and libraries and packages with Python 2, you can. You just have to choose between:

                                                                              • Use an older version of the project with no more bugfixes or security support, or
                                                                              • Form a group of like-minded people to do your own bugfixes and security support

                                                                              Way way back in the early days of Django when the “magic-removal” effort rewrote the ORM (between Django 0.91 and 0.95), creating a big backwards-incompatible jump for projects to get over, I and some other people took the second option: even after the core Django team wasn’t supporting Django 0.91 anymore, we maintained a Django 0.91 branch for a while in order to gain more time to migrate. It’s open source and you can do that.

                                                                              You can’t even look at a Python 2 user without the insinuation that they must be responsible for the entirety of their ecosystem.

                                                                              I could just as easily flip this around and say you can’t look at someone who develops on Python 3 without insinuating they owe you their time and effort to also maintain a Python 2 version of whatever they build.

                                                                              When you say “Python 2 is still supported”, you seem content to mean it in the specific narrow sense of “there is at least one group still maintaining a Python 2 interpreter”. What I’ve told you many times now is that when people hear “Python 2 is still supported”, what they will expect that to mean is a wider sense of popular third-party packages still maintaining Python 2 compatibility. I have concerns about that, because I maintain third-party packages and I’d rather not have a stream of people showing up in my bug trackers and getting angry because they heard “Python 2 is still supported” and thought that would mean I’ll maintain packages for them to use on Python 2.

                                                                              Frankly, I think that you show your hand when you say “really important packages like NumPy/SciPy.” That’s the direction that you want Python to go in.

                                                                              NumPy and SciPy are important packages. I don’t personally use them – as I’ve mentioned now multiple times in these discussions with you, I’m a Django guy, and my main use case for Python is as a backend web development lanaguage – but I know that there’s a large segment of the Python community who rely heavily on the numeric/scientific stack to do their work. I don’t see how anyone could look at the Python community and not come to that conclusion. So regardless of whether I use them or not, NumPy and SciPy are important packages. Just as Jupyter (née IPython) notebooks are important, even though I don’t personally use them. Just as the ML/AI packages are important even though I don’t use them. Just as Flask and SQLAlchemy are important packages, even though I don’t use them. Python’s continued success as a language comes from the large community of people using it for different things. The fact that there are large numbers of people using Python for not-my-use-case with not-the-libraries-I-use is a really good thing!

                                                                              Anyway. You’re angry at what you see as the marginalization of alternative Python implementations, and especially the one you personally are most involved with (PyPy). That’s very clear (in fact, you’ve even said that pretty explicitly). But you’re so angry about it that you’re lashing out at random people on the internet, and either ignoring or misinterpreting what they say.

                                                                              And on one level, I get it. The alternative implementation story has been rough! Django used to support IronPython and Jython, for example, and we treated it as a release blocker if we had compatibility issues with them. But at a certain point they started falling further and further behind CPython, and eventually it was clear that it would be a long time before they caught up, if ever. That’s sad – it used to be a great way to sneak Django into a Java or Windows shop – but I don’t believe Django has the resources to keep supporting such widely-diverged platforms long-term. If and when Jython and/or IronPython catch up to CPython (and both of them seem to have Python 3 compatibility as a goal), I’d be happy to push for Django to maintain compatibility with them again.

                                                                              But on another level, you’re not really helping your case when you treat people as your enemy for no longer supporting Python 2 in their packages. If I didn’t already have a long history of knowing other PyPy people, for example, I’d be coming away with a pretty negative view of the project from my interactions with you.

                                                                          2. 10

                                                                            also Twisted

                                                                            This document is obsolete. Twisted has been ported to Python 3 (jettisonning a few unmaintained modules in the process). The next release will require Python 3, likely 3.5+.

                                                                            1. 2

                                                                              Twisted did take about 15 million years to port, though, so, it does speak to how difficult porting can be. They did start a long time ago, and also have pretty much finished a while ago as well, so, anyone who hasn’t started should’ve kinda seen this (the end-of-life for Python 2) coming.

                                                                              1. 7

                                                                                Yeah, it was a huge project, and particularly difficult because Twisted is exactly the sort of code that is most difficult to port: it sits at the divide between bytes and str. Something like a Django app is much easier to port — both because of the solid groundwork laid by Django itself, and also because the bulk of a web app’s logic is only concerned with text.

                                                                                I do think that the OP undersells how unevenly the pain of the Python 3 transition has been distributed. It has been particularly rough for library maintainers, who often want to retain compatibility for a long time to enable their users to migrate. The end result has been a slow and painful transition with the corresponding toll on the Python community.

                                                                                I would really like to see a blameless post-mortem of the Python 3 transition. Surely we can learn some lessons from this process that apply to development going forward.

                                                                                1. 7

                                                                                  I do think that the OP undersells how unevenly the pain of the Python 3 transition has been distributed. It has been particularly rough for library maintainers, who often want to retain compatibility for a long time to enable their users to migrate.

                                                                                  Assuming by “OP” you mean me (author of the blog post this whole thread is about), I don’t think I’m “underselling”. I was just trying to write about a bunch of different thoughts and perspectives that I think either haven’t been covered, or haven’t been covered enough, in the copious literature on the Python 2/3 transition.

                                                                                  Something like a Django app is much easier to port — both because of the solid groundwork laid by Django itself

                                                                                  This is sort of what I was getting at, though. I remember the days of Django’s “unicode branch” and how much work it took to get Django to a point where Django apps mostly didn’t have to worry about that stuff. As the saying goes, a lot of effort went into making that look effortless, and Django’s early experiences strongly influenced my later feelings about the right way to handle strings.

                                                                                  1. 4

                                                                                    Assuming by “OP” you mean me (author of the blog post this whole thread is about), I don’t think I’m “underselling”.

                                                                                    Perhaps not the best word choice on my part.

                                                                                    I do think that the experience of making the transition varies a lot, and that may explain some of the heat in this thread and the discussion in general. We’re all frustrated by how things have dragged out.

                                                                                    As the saying goes, a lot of effort went into making that look effortless, and Django’s early experiences strongly influenced my later feelings about the right way to handle strings.

                                                                                    Yes, exactly! A post-mortem should focus on what Python could have done to make the transition work this way for everyone.

                                                                              2. 2

                                                                                Sure. The links I used fail to actually tell the story that I wanted to tell. There was a point in time at which most of Twisted was Python 2 and 2to3 wasn’t capable of making up the difference. At that time, people often asked for Python 3 support. However, Python 3 support was only happening through paid work, and community members were not spending time on the porting effort. Eventually, as you say, many modules were simply removed entirely rather than being ported.

                                                                                This is the process that I wanted to emphasize as happening repeatedly all over the Python ecosystem. Every library maintainer was asked to put in massive amount of effort, to the point where external funding was considered necessary to get things done. But this effort was only worthwhile because Python 3 supposedly was better than Python 2.

                                                                              3. 7

                                                                                My argument, in ten words or less, is: Okay, yes, but must we have changed “2” to “3”?

                                                                                I think that misses the point the author is trying to make. He argues that the maintainers of python ARE moving from 2 to 3. What we/you must do in this context is either assume the maintenance burden of a forked python 2 or migrate to python 3. You’re welcome to do neither as well, but of course any bugs related to your ecosystem will go unpatched.

                                                                                1. 4

                                                                                  People had already, as you pointed out, survived having to change with to be reserved and then a keyword, and that change was far more intrusive than anything in the 2-to-3 changelist.

                                                                                  Sorry but just to clarify: do you genuinely think that a keyword becoming reserved is a more intrusive change than switching to Unicode strings and explicit conversion to and from Unicode strings and byte sequences?

                                                                                  1. 0

                                                                                    As you’ve phrased it, no. However, what you’ve described is not what actually happened. Python 2 supports the Unicode sandwich technique just as well as Python 3, and has the unicode type. I would rephrase your question as, “Do I genuinely think that a keyword becoming reserved is a more intrusive change than renaming unicode and str to str and bytes respectively?” And the answer is yes, I do.

                                                                                    1. 12

                                                                                      From your link:

                                                                                      So Python 2’s pain is deferred: you think your program is correct, and find out later that it fails with exotic characters.

                                                                                      With Python 3, your code fails immediately, so even if you are only handling ASCII, you have to explicitly deal with the difference between bytes and unicode.

                                                                                      I can’t believe that a “security professional” would countenance implicit, latent bugs like this on a fundamental data type. I’ve done the unicode sandwich for years and write implicitly statically typed python as a matter of practice (which JITs nicely in PyPy without having to wonder “will it blend?”).

                                                                                      Of the several hundred engineers I’ve met and the dozens I’ve worked with daily for years, I can count the number of people who are aware of unicode/str implicit mixing on one hand.

                                                                                      You’d honestly preserve a constant subtle footgun than explicitly expose the incompatible boundaries so they’re dealt with early on and not left as a landmine to appear years later? Wow