Tomorrow seems to be a very bad day for all those poor souls, who didn’t have time/resources to switch to py3 yet.
Fortunately enough it can be easily fixed with pip<21 but it will definitely add additional grey hairs to some heads.
As one of those poor souls, thanks. We have eight years of legacy code that Just Works and so seldom gets touched, and a major 3rd party framework dependency that hasn’t updated to Python 3 either. We just got permission and funding to form a new engineering sub-group to try to deal with this sort of thing, and upper management is already implicitly co-opting it to chase new shinies.
Python 3.0 was released in 2008. I personally find it hard to feel sympathy for anyone who couldn’t find time in the last twelve years to update their code, especially if it’s code they are still using today. Even more so for anyone who intentionally started a Python 2 project after the 3.0 ecosystem had matured.
Python 2.7 was released in 2010. Python 3.3 in 2012. Python 2.6 last release was in 2013. Only from this date people could easily release stuff compatible with both Python 2 and Python 3. You may also want to take into consideration the end of support date of some of the distributions shipping Python 2.6 and not Python 2.7 (like Debian Squeeze, 2016).
I am not saying that 8 years is too fast, but Python 3.0 release date is mostly irrelevant as the ecosystem didn’t use it.
Python 3.0 was not something you wanted to use; it took several releases before Python 3 was really ready for people to write programs on. Then it took longer for good versions of Python 3 to propagate into distributions (especially long term distributions), and then it took longer for people to port packages and libraries to Python 3, and so on and so forth. It has definitely not been twelve years since the ecosystem matured.
Some people do enough with Python that it’s sensible for them to build and maintain their own Python infrastructure, so always had the latest Python 3. Many people do not and so used supplied Python versions, and may well have stable Python code that just works and they haven’t touched in years (perhaps because they are script-level infrastructure that just sits there working, instead of production frontend things that are under constant evolution because business needs keep changing).
Some of our toolchain broke in the last few weeks. We ported to python3 ages ago, but chunks of infrastructure still support both, and some even still default to 2. The virtualenv binary in Ubuntu 18.02 does that; and that’s a still-supported Ubuntu version, and the default runner for GitHub CI.
I think python2-related pain will continue for years to come even for people who have done the due diligence on their own code.
Small tip regarding virtualenv: Since python 3.3 virtualenv comes bundled as the venv module in python, so you can just use python -m venv instead of virtualenv, then you are certain it matches the python version you are using.
virtualenv has some nice features which do not exist for venv. One of the examples is activate_this.py script, which can be used for configuration of remote environment, similar to what pytest_cloud does.
virtualenv has some nice features which do not exist for venv
Huh, thanks for pointing that out. I haven’t been writing so much Python in the last few years, and I totally thought venv and virtualenv were the same thing.
Consider, at a minimum, the existence of PyPy; PyPy’s own position is that PyPy will support Python 2.7 forever because PyPy is written in RPython, a strict subset of Python 2.7.
Sympathy is not required; what you’re missing out on is an understanding that Python is not wholly under control of the Python Software Foundation. By repeatedly neglecting PyPy, the PSF has effectively forced them to create their own parallel Python 2 infrastructure; when PyPI finally makes changes which prevent Python 2 code from deploying, then we may see PyPy grow even more tooling and possibly even services to compensate.
It is easy for me to recognize in your words an inkling of contempt for Python 2 users.
Every time you hop into one of these threads, you frame it in a way which implies you think various entities are obligated to maintain a Python 2 interpreter, infrastructure for supporting Python 2 interpreters, and versions of third-party packages which stay compatible with Python 2, for all of eternity.
Judging from that last thread, you seem to think I am one of the people who has that obligation. Could you please, clearly, state to me the nature of this obligation – is its basis legal? moral? something else? – along with its origin and the means by which you assume the right to impose it on me.
I ask because I cannot begin to fathom where such an obligation would come from, nor do I understand why you insist on labeling it “contempt” when other people choose not to maintain software for you, in the exact form you personally prefer, for free, forever, anymore.
Your sympathy, including any effort or obligation that you might imagine, is not required. I don’t know how to put it any more clearly to you: You have ended up on the winning side of a political contest within the PSF, and you are antagonizing members of the community who lost for no other reason than that you want the political divide to deepen.
Maybe, to get some perspective, try replacing “Python 2” with “Perl 5” and “Python 3” with “Raku”; that particular community resolved their political divide recently and stopped trying to replace each other. Another option for perspective: You talk about “these threads”; what are these threads for, exactly? I didn’t leave a top-level comment on this comment thread; I didn’t summon you for the explicit purpose of flamewar.
Finally, why not reread the linked thread? I not only was clearly the loser in that discussion, but I also explained that I personally am not permanently tied to Python 2, and that I’m trying to leave the ecosystem altogether in order to avoid these political problems. Your proposed idea of obligation towards me is completely imagined and meant to make you seem like a victim.
Here are some quotes which I think display contempt towards Python 2 and its users, from the previous thread (including your original post) and also the thread before that one:
If PyPy wants to internally maintain the interpreter they use to bootstrap, I don’t care one way or another. But if PyPy wants that to also turn into broad advertisement of a supported Python 2 interpreter for general use, I hope they’d consider the effect it will have on other people.
Want to keep python 2 alive? Step up and do it.
What do you propose they do then? Extend Python 2 support forever and let Python 2 slow down Python 3 development for all time?
That’s them choosing and forever staying on a specific dependency. … Is it really that difficult for Python programmers to rewrite one Python program in the newer version of Python? … Seems more fair for the project that wants the dependency to be the one reworking it.
The PyPy project, for example, is currently dependent on a Python 2 interpreter to bootstrap and so will be maintaining their own either for as long as PyPy exists, or for as long as it takes to migrate to bootstrapping on Python 3 (which they seem to think is either not feasible, or not something they want to do).
He’s having a tantrum. … If you’re not on 3, it’s either a big ball of mud that should’ve been incrementally rewritten/rearchitected (thus exposing bad design) or you expected an ecosystem to stay in stasis forever.
I’m not going to even bother with your “mother loved you best” vis a vis PyPy.
You’re so wrapped up in inventing enemies that heap contempt on you, but it’s just fellow engineers raising their eyebrows at someone being overly dramatic. Lol contempt. 😂😂😂
If I didn’t already have a long history of knowing other PyPy people, for example, I’d be coming away with a pretty negative view of the project from my interactions with you.
What emotional word would you use to describe the timbre of these attitudes? None of this has to do with maintainership; I don’t think that you maintain any packages which I directly require. I’m not asking for any programming effort from you. Indeed, if you’re not a CPython core developer either, then you don’t have the ability to work on this; you are also a bystander. I don’t want sympathy; I want empathy.
You have ended up on the winning side of a political contest within the PSF, and you are antagonizing members of the community who lost for no other reason than that you want the political divide to deepen.
And this is where the problem lies. Your behavior in the previous thread, and here, makes clear that your approach is to insult, attack, or otherwise insinuate evil motives to anyone who disagrees with you.
Here are some quotes which I think display contempt towards Python 2 and its users
First of all, it’s not exactly courteous to mix and match quotes from multiple users without sourcing them to who said each one. If anyone wants to click through to the actual thread, they’ll find a rather different picture of, say, my engagement with you. But let’s be clear about this “contempt”.
In the original post, I said:
The PyPy project, for example, is currently dependent on a Python 2 interpreter to bootstrap and so will be maintaining their own either for as long as PyPy exists, or for as long as it takes to migrate to bootstrapping on Python 3 (which they seem to think is either not feasible, or not something they want to do).
You quoted this and replied:
This quote is emblematic of the contempt that you display towards Python users.
I remain confused as to what was contemptuous about that. You yourself have confirmed that PyPy is in fact dependent on a Python 2 interpreter, and your own comments seem to indicate there is no plan to migrate away from that dependency. It’s simply a statement of fact. And the context of the quote you pulled was a section exploring the difference between “Python 2” the interpreter, and “Python 2” the ecosystem of third-party packages. Here’s the full context:
Unfortunately for that argument, Python 2 was much more than just the interpreter. It was also a large ecosystem of packages people used with the interpreter, and a community of people who maintained and contributed to those packages. I don’t doubt the PyPy team are willing to maintain a Python 2 interpreter, and that people who don’t want to port to Python 3 could switch to the PyPy project’s interpreter in order to have a supported Python 2 interpreter. But a lot of those people would continue to use other packages, too, and as far as I’m aware the PyPy team hasn’t also volunteered to maintain Python 2 versions of all those packages.
So there’s a sense in which I want to push back against that messaging from PyPy folks and other groups who say they’ll maintain “Python 2” for years to come, but really just mean they’ll maintain an interpreter. If they keep loudly announcing “don’t listen to the Python core team, Python 2 is still supported”, they’ll be creating additional burdens for a lot of other people: end users are going to go file bug reports and other support requests to third-party projects that no longer support Python 2, because they heard “Python 2 is still supported”, and thus will feel entitled to have their favorite packages still work.
Even if all those requests get immediately closed with “this project doesn’t support Python 2 anymore”, it’s still going to take up the time of maintainers, and it’s going to make the people who file the requests angry because now they’ll feel someone must be lying to them — either Python 2 is dead or it isn’t! — and they’ll probably take that anger out on whatever target happens to be handy. Which is not going to be good.
This is why I made comments asking you to consider the effect of your preferred stance on other people (i.e., on package maintainers). This is why I repeated my point in the comments of the previous thread, that an interpreter is a necessary but not sufficient condition for saying “Python 2 is still supported”. I don’t think these are controversial statements, but apparently you do. I don’t understand why.
I also still don’t understand comments of yours like this one:
Frankly, I think that you show your hand when you say “really important packages like NumPy/SciPy.” That’s the direction that you want Python to go in.
Again, this is just a statement of fact. There are a lot of people using Python for a lot of use cases, and many of those use cases are dependent on certain domain-specific libraries. As I said in full:
So regardless of whether I use them or not, NumPy and SciPy are important packages. Just as Jupyter (née IPython) notebooks are important, even though I don’t personally use them. Just as the ML/AI packages are important even though I don’t use them. Just as Flask and SQLAlchemy are important packages, even though I don’t use them. Python’s continued success as a language comes from the large community of people using it for different things. The fact that there are large numbers of people using Python for not-my-use-case with not-the-libraries-I-use is a really good thing!
Your words certainly imply you think it’s a bad thing that there are, for example, people using NumPy and SciPy, or at least that you think that’s a bad direction for Python to go in. I do not understand why, and you’ve offered no explanation other than to hand-wave it as “contempt” and “denigration”.
But really the thing I do not understand is this:
You have ended up on the winning side of a political contest within the PSF
You seem to think that “the PSF” and/or some other group of people or entities in the Python world are your enemy, because they chose to move to Python 3 and to stop dedicating their own time and resources to maintaining compatibility with and support for Python 2. The only way that this would make any sense is if those entities had some sort of obligation, to you or to others, to continue maintaining compatibility with and support for Python 2. Hence I have asked you for an explanation of the nature and origin of that obligation so that I can try to understand the real root of why you seem to be so angry about this.
Admittedly I don’t have high hopes for getting such an explanation, given what happened last time around, but maybe this time?
Your behavior in the previous thread, and here, makes clear that your approach is to insult, attack, or otherwise insinuate evil motives to anyone who disagrees with you.
As Corbin has said themselves multiple times, they are not a nice person. So unfortunately you can’t really expect anything better than this.
From my OpenStack experience – many automated gates could go south, because they could do something like: pip install pip --upgrade hence dropping support for py2. I know, that whomever is involved in this conundrum, should know better and should introduce some checks. But I also know, that we’re all humans, hence prone to make errors.
pip install pip --upgrade should still work, unless the pip team screwed something up.
When you upload something to PyPI, you can specify a minimal support Python version. So Python 2.7 will get the latest version that still supports Python 2.
And indeed, if you go to https://pypi.org/project/pip/ you will see “Requires: Python >= 3.6”, so I expect things will Just Work for most Python 2 users.
Python 2 support has been dropped by pip the Python Packaging Authority (PyPA) at the Python Packaging Index (PyPI), which is the default configuration for every distribution of pip I’m aware of. If you have critical dependencies on Python 2 packages and are unwilling to migrate to Python 3, set up your own package index or pull the libraries you depend on directly into the vcs for your legacy project (which is definitely more work than migrating to Python 3, but is a choice you have).
Another easier alternative would be to pull the libraries you depend on directly into the vcs for your legacy project.
This is a change to pip, not to pypi. You can still use an older version of pip on Python 2 to install packages from PyPI. (That might change in the future, but it doesn’t seem to be under consideration in the maintainers threads on the issue.)
Vendoring dependencies is pretty mechanical. Migrating to Python 3 is only partially mechanical. So I’m not sure why you would say the former is more work. It doesn’t seem like it to me. By far.
I wrote the parenthetical statement in reference to setting up your own index, then went back to add the alternative option (of vendoring dependencies), so that was an unintended misstatement. Thanks for the clarification; you are completely correct. Original comment now reflects the correction above.
Interesting, okay. Depending on circumstances, setting up your own index could also be easier than migrating to Python 3 though too. Migrating to Python 3 can be exceptionally difficult. I’ve lived through it.
I talked a friend through the decision at his $work, and they decided the site deployment wasn’t large enough to warrant going in that direction, since it presented an ongoing cost for onboarding future people and maintaining infrastructure. They ultimately pulled all libraries into their own repo, not even vendoring, after going through the process and discovering none were being actively maintained anyway.
Python 2 support has been dropped by pip the Python Packaging Authority (PyPA) at the Python Packaging Index (PyPI)
I don’t think that this statement is correct. Pip dropping support for Python 2.7 doesn’t inherently have any implications for what packages can be uploaded or downloaded from PyPI. I’ve not heard a peep about dropping support for any version of Python on PyPI.
Vendoring your unmaintained Python 2.7 dependencies may be useful for other reasons: while you’ll still be able to use Pip 2.3.x for a while it will eventually atrophy, like everything else in the 2.7 ecosystem. Vendoring may ease any sustaining engineering you do, and it’ll help avoid use of an unmaintained client application. However, the version of Pip shipped with your Linux distribution will doubtless continue to be supported (meaning: receive security updates to the TLS stack) for years, and PyPI is unlikely to break it.
Tomorrow seems to be a very bad day for all those poor souls, who didn’t have time/resources to switch to py3 yet. Fortunately enough it can be easily fixed with
pip<21but it will definitely add additional grey hairs to some heads.As one of those poor souls, thanks. We have eight years of legacy code that Just Works and so seldom gets touched, and a major 3rd party framework dependency that hasn’t updated to Python 3 either. We just got permission and funding to form a new engineering sub-group to try to deal with this sort of thing, and upper management is already implicitly co-opting it to chase new shinies.
Python 3.0 was released in 2008. I personally find it hard to feel sympathy for anyone who couldn’t find time in the last twelve years to update their code, especially if it’s code they are still using today. Even more so for anyone who intentionally started a Python 2 project after the 3.0 ecosystem had matured.
Python 2.7 was released in 2010. Python 3.3 in 2012. Python 2.6 last release was in 2013. Only from this date people could easily release stuff compatible with both Python 2 and Python 3. You may also want to take into consideration the end of support date of some of the distributions shipping Python 2.6 and not Python 2.7 (like Debian Squeeze, 2016).
I am not saying that 8 years is too fast, but Python 3.0 release date is mostly irrelevant as the ecosystem didn’t use it.
Python 3.0 was not something you wanted to use; it took several releases before Python 3 was really ready for people to write programs on. Then it took longer for good versions of Python 3 to propagate into distributions (especially long term distributions), and then it took longer for people to port packages and libraries to Python 3, and so on and so forth. It has definitely not been twelve years since the ecosystem matured.
Some people do enough with Python that it’s sensible for them to build and maintain their own Python infrastructure, so always had the latest Python 3. Many people do not and so used supplied Python versions, and may well have stable Python code that just works and they haven’t touched in years (perhaps because they are script-level infrastructure that just sits there working, instead of production frontend things that are under constant evolution because business needs keep changing).
Some of our toolchain broke in the last few weeks. We ported to python3 ages ago, but chunks of infrastructure still support both, and some even still default to 2. The virtualenv binary in Ubuntu 18.02 does that; and that’s a still-supported Ubuntu version, and the default runner for GitHub CI.
I think python2-related pain will continue for years to come even for people who have done the due diligence on their own code.
Small tip regarding
virtualenv: Since python 3.3 virtualenv comes bundled as thevenvmodule in python, so you can just usepython -m venvinstead ofvirtualenv, then you are certain it matches the python version you are using.virtualenvhas some nice features which do not exist forvenv. One of the examples isactivate_this.pyscript, which can be used for configuration of remote environment, similar to what pytest_cloud does.Huh, thanks for pointing that out. I haven’t been writing so much Python in the last few years, and I totally thought
venvandvirtualenvwere the same thing.Consider, at a minimum, the existence of PyPy; PyPy’s own position is that PyPy will support Python 2.7 forever because PyPy is written in RPython, a strict subset of Python 2.7.
Sympathy is not required; what you’re missing out on is an understanding that Python is not wholly under control of the Python Software Foundation. By repeatedly neglecting PyPy, the PSF has effectively forced them to create their own parallel Python 2 infrastructure; when PyPI finally makes changes which prevent Python 2 code from deploying, then we may see PyPy grow even more tooling and possibly even services to compensate.
It is easy for me to recognize in your words an inkling of contempt for Python 2 users.
Every time you hop into one of these threads, you frame it in a way which implies you think various entities are obligated to maintain a Python 2 interpreter, infrastructure for supporting Python 2 interpreters, and versions of third-party packages which stay compatible with Python 2, for all of eternity.
Judging from that last thread, you seem to think I am one of the people who has that obligation. Could you please, clearly, state to me the nature of this obligation – is its basis legal? moral? something else? – along with its origin and the means by which you assume the right to impose it on me.
I ask because I cannot begin to fathom where such an obligation would come from, nor do I understand why you insist on labeling it “contempt” when other people choose not to maintain software for you, in the exact form you personally prefer, for free, forever, anymore.
Your sympathy, including any effort or obligation that you might imagine, is not required. I don’t know how to put it any more clearly to you: You have ended up on the winning side of a political contest within the PSF, and you are antagonizing members of the community who lost for no other reason than that you want the political divide to deepen.
Maybe, to get some perspective, try replacing “Python 2” with “Perl 5” and “Python 3” with “Raku”; that particular community resolved their political divide recently and stopped trying to replace each other. Another option for perspective: You talk about “these threads”; what are these threads for, exactly? I didn’t leave a top-level comment on this comment thread; I didn’t summon you for the explicit purpose of flamewar.
Finally, why not reread the linked thread? I not only was clearly the loser in that discussion, but I also explained that I personally am not permanently tied to Python 2, and that I’m trying to leave the ecosystem altogether in order to avoid these political problems. Your proposed idea of obligation towards me is completely imagined and meant to make you seem like a victim.
Here are some quotes which I think display contempt towards Python 2 and its users, from the previous thread (including your original post) and also the thread before that one:
What emotional word would you use to describe the timbre of these attitudes? None of this has to do with maintainership; I don’t think that you maintain any packages which I directly require. I’m not asking for any programming effort from you. Indeed, if you’re not a CPython core developer either, then you don’t have the ability to work on this; you are also a bystander. I don’t want sympathy; I want empathy.
And this is where the problem lies. Your behavior in the previous thread, and here, makes clear that your approach is to insult, attack, or otherwise insinuate evil motives to anyone who disagrees with you.
First of all, it’s not exactly courteous to mix and match quotes from multiple users without sourcing them to who said each one. If anyone wants to click through to the actual thread, they’ll find a rather different picture of, say, my engagement with you. But let’s be clear about this “contempt”.
In the original post, I said:
You quoted this and replied:
I remain confused as to what was contemptuous about that. You yourself have confirmed that PyPy is in fact dependent on a Python 2 interpreter, and your own comments seem to indicate there is no plan to migrate away from that dependency. It’s simply a statement of fact. And the context of the quote you pulled was a section exploring the difference between “Python 2” the interpreter, and “Python 2” the ecosystem of third-party packages. Here’s the full context:
This is why I made comments asking you to consider the effect of your preferred stance on other people (i.e., on package maintainers). This is why I repeated my point in the comments of the previous thread, that an interpreter is a necessary but not sufficient condition for saying “Python 2 is still supported”. I don’t think these are controversial statements, but apparently you do. I don’t understand why.
I also still don’t understand comments of yours like this one:
Again, this is just a statement of fact. There are a lot of people using Python for a lot of use cases, and many of those use cases are dependent on certain domain-specific libraries. As I said in full:
Your words certainly imply you think it’s a bad thing that there are, for example, people using NumPy and SciPy, or at least that you think that’s a bad direction for Python to go in. I do not understand why, and you’ve offered no explanation other than to hand-wave it as “contempt” and “denigration”.
But really the thing I do not understand is this:
You seem to think that “the PSF” and/or some other group of people or entities in the Python world are your enemy, because they chose to move to Python 3 and to stop dedicating their own time and resources to maintaining compatibility with and support for Python 2. The only way that this would make any sense is if those entities had some sort of obligation, to you or to others, to continue maintaining compatibility with and support for Python 2. Hence I have asked you for an explanation of the nature and origin of that obligation so that I can try to understand the real root of why you seem to be so angry about this.
Admittedly I don’t have high hopes for getting such an explanation, given what happened last time around, but maybe this time?
As Corbin has said themselves multiple times, they are not a nice person. So unfortunately you can’t really expect anything better than this.
Why will tomorrow be a bad day? pip will continue to work. They’re just stopping releasing updates.
From my OpenStack experience – many automated gates could go south, because they could do something like:
pip install pip --upgradehence dropping support for py2. I know, that whomever is involved in this conundrum, should know better and should introduce some checks. But I also know, that we’re all humans, hence prone to make errors.pip install pip --upgradeshould still work, unless the pip team screwed something up.When you upload something to PyPI, you can specify a minimal support Python version. So Python 2.7 will get the latest version that still supports Python 2.
And indeed, if you go to https://pypi.org/project/pip/ you will see “Requires: Python >= 3.6”, so I expect things will Just Work for most Python 2 users.
Python 2 support has been dropped by
pipthe Python Packaging Authority (PyPA) at the Python Packaging Index (PyPI), which is the default configuration for every distribution of pip I’m aware of. If you have critical dependencies on Python 2 packages and are unwilling to migrate to Python 3, set up your own package indexor pull the libraries you depend on directly into the vcs for your legacy project(which is definitely more work than migrating to Python 3, but is a choice you have).Another easier alternative would be to pull the libraries you depend on directly into the vcs for your legacy project.
This is a change to
pip, not topypi. You can still use an older version ofpipon Python 2 to install packages from PyPI. (That might change in the future, but it doesn’t seem to be under consideration in the maintainers threads on the issue.)That wasn’t what I read the change to mean, so absolutely needed your clarification. Cheers.
Vendoring dependencies is pretty mechanical. Migrating to Python 3 is only partially mechanical. So I’m not sure why you would say the former is more work. It doesn’t seem like it to me. By far.
I wrote the parenthetical statement in reference to setting up your own index, then went back to add the alternative option (of vendoring dependencies), so that was an unintended misstatement. Thanks for the clarification; you are completely correct. Original comment now reflects the correction above.
Interesting, okay. Depending on circumstances, setting up your own index could also be easier than migrating to Python 3 though too. Migrating to Python 3 can be exceptionally difficult. I’ve lived through it.
I talked a friend through the decision at his $work, and they decided the site deployment wasn’t large enough to warrant going in that direction, since it presented an ongoing cost for onboarding future people and maintaining infrastructure. They ultimately pulled all libraries into their own repo, not even vendoring, after going through the process and discovering none were being actively maintained anyway.
I don’t think that this statement is correct. Pip dropping support for Python 2.7 doesn’t inherently have any implications for what packages can be uploaded or downloaded from PyPI. I’ve not heard a peep about dropping support for any version of Python on PyPI.
Vendoring your unmaintained Python 2.7 dependencies may be useful for other reasons: while you’ll still be able to use Pip 2.3.x for a while it will eventually atrophy, like everything else in the 2.7 ecosystem. Vendoring may ease any sustaining engineering you do, and it’ll help avoid use of an unmaintained client application. However, the version of Pip shipped with your Linux distribution will doubtless continue to be supported (meaning: receive security updates to the TLS stack) for years, and PyPI is unlikely to break it.