1. 27

It’s really disappointing to me that the core python developers hear stories like this but don’t want to find ways to make it easier to port codebases to python3.


  2. 10

    I’m curious what the python3 developers expected to happen. Was converting to python3 supposed to be trivial? Was somebody supposed to reimplement a newer, better twisted for python3? Were app developers supposed to toss twisted aside? Or is everything going according to plan? Was updating twisted supposed to take years of effort? It kind of looks like a debacle from the outside, but I’m constantly assured everything is just peachy, so I don’t know.

    1. 8

      This post is a good reference about why Python 3 exists. He also talks about the smorgasbord of features in this post. Brett’s a Python core developer and an all-around great person. The vast majority of the most popular libraries (e.g., requests, pycrypto) and frameworks (e.g., django, SQLAlchemy) have already been ported to Python 3 and are actively maintained there. You can see the list of progress here.

      The important thing to note is that Twisted is a relatively small subculture within the Python community. A lot of Python programmers — myself included — won’t touch it because it’s overly complex and can be so confusing (especially for newbies to a codebase). But Twisted is used by a lot of popular software tools and libraries, which is why porting it is important in general.

      I’d say the more interesting topic is the asyncio library, which was made standard in Python 3.3. It’s a grand unification of all async and event-loop behavior in Python. I think it’s fair to argue that most of the core facilities of Twisted (like Deferreds) are no longer relevant and should be deleted. What’s left in Twisted that’s extremely valuable is its support for a wide variety of very complex wire protocols.

      1. 6

        I did a lot of work with Twisted a while back, and I remember feeling like it was really powerful, but also a really big world unto itself, just tons of code and lots to get used to. Also pretty unfamiliar for someone used to either straightforward Python or multithreaded C. I’m not surprised that it’s taken this long to port it - it was always going to be hard and I never felt like the incentives were there to fund the work.

        Add to that the point made at the end of the article, that although Twisted does a lot more than just async execution, that’s kind of the core of it, and now there’s asyncio in the standard library, which you can use without adopting the rest of the Twisted world.

        1. 2

          I am also curious about this. I have only seen essays by the detractors, not the supporters, and in a quick search right now all I could find from python 3 enthusiasts were guides to porting… nothing about why anyone would want to.

          Let me know if you find anything.

          1. 5

            You “want” to because the 2.x series is no longer going to receive updates.

            That’s probably as much reasoning as you are going to get.

            1. 5

              There are a few nice features, but nothing worth the pain. Keyword-only arguments are particularly nice for library authors.

              1. 1

                See the posts I link in the higher thread. The gist from Brett Cannon is:

                We knew Python’s popularity was on an upward trend and we hoped its growth would continue … But this also meant that if we were going to fix any design mistakes and help continue the language’s popularity, we needed to do it now rather than later.

            2. 6

              Meanwhile, in Ruby, everybody seems to have just quietly kept up-to-date without issue.

              I am rather curious how the core language folks have managed to design a major update that people have actively run away from for seven years. Is it because the core team is off in space, because the community is derpy, because Python is just rather shite, or what?

              (I’m not a python person, so this is really baffling from the outside looking in.)

              1. 8

                [I’ve spent a few years programming in python professionally]

                It’s a bit of everything really. The initial python3 release seemed like a grab-bag of different breaking changes, some essential and some trivial, all rolled into one monolithic release. The unicode thing needed to happen, but the breakage of the print statement indicated that the core devs couldn’t tell the difference between a painful-yet-essential change and one which was just awkward and trivial.

                Most python devs I’ve worked with have expressed some level of discomfort with the python3 roll-out, most of it thematically similar to “it’s a painful migration with no benefit”.

                Meanwhile there was no sign of improvements people actually wanted, like lifting the GIL or improving VM performance.

                By contrast, I think Ruby did a better job because they seem to introduce one breaking change at a time, with a clear migration path into the new version, so the whole ecosystem can deal with the pain and move basically in lock-step.

                1. 3

                  I’ve been a Python programmer for almost 20 years, but haven’t used Python3 extensively until recently.

                  I disagree with your premise that community ran away from it. It took a while for it to be adopted, but at this point I would say that happened even though many still use 2.x series and will for a long time to come. It was always expected to take a while, but it may have taken longer because carrots and stick approach was initially light on carrots and (still is) lacking sticks.

                  First couple of 3.x releases were mainly about setting up new, cleaner infrastructure and less about features compelling enough to make you work through new incompatible changes. It didn’t help that some of the nice things they introduced were back-ported to 2.x and that you couldn’t “easily” evolve your 2.x code to 3.x until versions 2.7 and 3.3. Not understanding that this will be important was certainly a mistake.

                  Additionally most library authors who wrote 3.x version of libraries still tried to be nice and also support 2.x (hence, no sticks). I do too, but probably won’t in the future. It wasn’t until fairly recently that I’ve started to see 3.x-only versions of Python packages.

                  Most projects are not green field and when they are, they will still use other people’s code (libraries). It would help if most of us were really unhappy with 2.x versions, but we weren’t so it was easier to wait until major pieces of your infrastructure moved then to find alternatives or write your own. For the most part this has been done now and in the mean time we also learned to separate services better so it is easier to move to Python3 incrementally which is what I see has very much happened.

                2. 1

                  On average, she spent two hours a day for a year and a half working on it.

                  And she’s only half way there. Now multiply this for the number of complex Python 2 projects out there and you’ll start to see the madness of breaking backwards compatibility.

                  1. 2

                    There’s not really a way to fix the core bytes/unicode ambiguity problem in Python 2 without breaking backwards compatibility. It’s a necessary step to get the benefit.

                    1. 1

                      Where there’s a will, there’s a way.

                      In this case, the core developers did what was easier for them, disregarding the amount of work they generate for their users. It seems to be a common practice in the Python community - Django breaks compatibility every minor version.