I’ve always had the privilege of working places where I am not responsible for more code than I can maintain, which is why I’m always shocked at the amount of people in the opposite situation: having more code than you can maintain. I’m not referring to Python itself (inheriting a decades-old open source project is significantly different). I’m just referring to *users *of python who beg and plead for backwards-compatibility to never be broken so they don’t have to ever change their code. Just change it! That’s your job!
If you have leadership who is refusing to hiring extra devs to save you from drowning under the burden of maintenance, that’s one thing. But IMO it’s just bad engineering to ignore the costs of maintenance, including maybe having to change an import or a function name before upgrading. And to offload the consequences of that mistake onto the open-source community really grinds my gears.
Hi… as the maintainer for a project that uses the cgi module, I am frustrated that I am now going to have to vendor FieldStorage vs just being able to use it from the standard library.
I even posted as such when this was proposed in 2019, but alas it did not make the cut. Shame really, as now those things are going to exist in countless different projects.
Also, the replacement (multipart) is still marked as “beta” on PyPi, and adds yet another dependency that I as a maintainer need to watch for/understand their upgrade cycles, make sure that their API doesn’t have breaking changes.
Anyway, disappointed that they are removing something that is still very useful. Surprisingly (or maybe not) the cgi module is still fairly heavily used in embedded systems/devices that want to run a quick Python script or two through a httpd of some sort without needing to run it full time or using a Python web server.
The problem with “pull in random pip library” is one day the maintainer disappears.
That in itself is understandable. But the time between that and people realizing, and the time between that and actually getting to “OK let’s fork it I suppose”, and to actually setting up a new maintainer can be years. Again, the maintainer can do what they want, but the knock-on effects exist, and it’s not for lack of people wanting to manage releases!
This is a common problem with Django-related projects. Django doesn’t break backwards compat super often, and loads of people stick on LTS things, but you end up with packages that are just stuck without fixes of the “rename this import” variety.
This is part of why I do work with Jazzband (to just fix stuff), but I generally believe that our future will be much nicer if packages with any sort of real usage end up in some system that guarantees that we can find new maintainers if needed.
Maybe we need python-contrib that just includes the things “most people use” (requests/attrs etc).
The problem with “pull in random pip library” is one day the maintainer disappears.
This problem doesn’t go away from having it in the standard library. The maintainer goes away but the standard library as a whole still has maintainers. Bugs in that bit don’t get fixed because no one left remembers how it works. Eventually there’s a critical security vulnerability and the rest of the maintainers realise that the last person who knows how to fix it left the project a decade ago.
If anything, it’s more obvious in a separate project if there are still active maintainers.
I’m not familiar with the maintenance of python’s stdlib in particular, but in the general case I’ve noticed a stark difference. For example, in ruby or Go or JS, you’ll be far more likely to run into bugs or other problems with 3rd party modules. Corporate backing, the sheer number of people relying on it, the official identity of the language being tied to it…. these cultural forces seem to make a difference.
C and C++ have much smaller standard libraries but in both there are parts of common implementations of the standard library that very few people understand and that haven’t been touched for over 10 years. Mostly they work and so no one needs to touch the code but if there ever is a bug then good luck
Actually there is a core difference, in which the standard library has its own maintainers so if you personally want to start maintaining a thing that is not getting love, there are contact points that are, in theory, available and have the keys to the kingdom.
This is colored by my personal experience, but usually the lack of maintenance becomes a problem not from security issues, but from slow API changes in related libraries. Most people who have a habit of jumping into unknown code can look at a thing and try to untangle the mess in a implementation-compatible way. But if they don’t have release rights that work doesn’t go anywhere.
EDIT: There are exceptional cases to the contrary, but what I see a lot of is not projects without willing maintainers, but projects in limbo because passing the torch is difficult without a whole flow for determining that.
If you vendor it you start with this problem, and worse because someone may actually be maintaining their vendorerd copy and you won’t even know! Better to maintain a library people can use than everyone maintain their own copy. That’s why we invented these systems :)
True in part (it doesn’t use the standard library version of optparse, but it still uses a copy of it, specifically because optparse was already deprecated when Click was created), but the point still stands: there are projects out there that will need to do something similar, or they’ll break.
tkinter could probably be moved outside of the standard library. Most distros package it separately anyway. I think the only reason it still gets bundled with Python is that it’d be nigh impossible to distribute from PyPI.
That, and also the fact that IDLE is a nice editor for teaching kids programming in Python, and I’d say a decent value-add in the ease of use sense (It comes with it’s own IDE!).
True! Though IDLE is desperately in need of some love to make it feel at least as friendly as VS Code. That doesn’t actually require much: maybe have Tkinter be less rubbish under MacOS, give a way to manage a project (even if it doesn’t understand version control, but for that, setuptools hooks exist), and maybe lean on jedi or and LSP daemon if it’s available?
The cgi module! Such nostalgia seeing it go. I doubt 99% of the people have used it in the last decade or two. But I always thought of it as viable quick and dirty way to put together a webUI for all sorts of tasks to be executed on demand on a controlled environment.
But in all fairness, utis has no use case I can think of that isn’t solved by http.server.
The tittle of this submission is a little misleading. “Finally”? Thid is a small set of long forgotten relics from past days. Perhaps accounting not more than dozens of users world wide.
I was kind of up in arms about this until I learned that the sources aren’t being removed. You’ll just need to vendor them to use them.
I DO understand this is additional work and nobody wants that, but I also think we all need to be a bit pragmatic about the fact that Python is HUGE and the people maintaining it only have so many hours in their lives :)
I’m glad the core team bit the bullet and is making this happen. I hope everyone affected can find their way to not sending too much vitriol upstream in response :)
I just meant: deprecating or removing parts of the standard library doesn’t seem like it has a big impact on the size/complexity. Still a big std lib, so is the breaking change worth it?
Every library that stays in needs ongoing maintenance in order to keep up with language changes and security compliance.
Also, with a toolset like Python which has enjoyed vast adoption, any change is a ‘breaking’ change for someone. The core team has to do the King Solomon thing and decide which course of action will benefit the most people.
Now, as much as I will ALWAYS have great fondness for Amiga IFF format libraries and uu-coding, these are tools that no longer serve the majority of the community’s needs.
So I think removing them from the standard distro but leaving them in Git so anyone who wants them can vendor them is a perfectly reasonable solution.
I’ve always had the privilege of working places where I am not responsible for more code than I can maintain, which is why I’m always shocked at the amount of people in the opposite situation: having more code than you can maintain. I’m not referring to Python itself (inheriting a decades-old open source project is significantly different). I’m just referring to *users *of python who beg and plead for backwards-compatibility to never be broken so they don’t have to ever change their code. Just change it! That’s your job!
If you have leadership who is refusing to hiring extra devs to save you from drowning under the burden of maintenance, that’s one thing. But IMO it’s just bad engineering to ignore the costs of maintenance, including maybe having to change an import or a function name before upgrading. And to offload the consequences of that mistake onto the open-source community really grinds my gears.
Waiting for the person who really really likes the cgi module and the optparse to comment on this.
Hi… as the maintainer for a project that uses the cgi module, I am frustrated that I am now going to have to vendor FieldStorage vs just being able to use it from the standard library.
I even posted as such when this was proposed in 2019, but alas it did not make the cut. Shame really, as now those things are going to exist in countless different projects.
Also, the replacement (multipart) is still marked as “beta” on PyPi, and adds yet another dependency that I as a maintainer need to watch for/understand their upgrade cycles, make sure that their API doesn’t have breaking changes.
Anyway, disappointed that they are removing something that is still very useful. Surprisingly (or maybe not) the cgi module is still fairly heavily used in embedded systems/devices that want to run a quick Python script or two through a httpd of some sort without needing to run it full time or using a Python web server.
Could they not go in a pip library?
The problem with “pull in random pip library” is one day the maintainer disappears.
That in itself is understandable. But the time between that and people realizing, and the time between that and actually getting to “OK let’s fork it I suppose”, and to actually setting up a new maintainer can be years. Again, the maintainer can do what they want, but the knock-on effects exist, and it’s not for lack of people wanting to manage releases!
This is a common problem with Django-related projects. Django doesn’t break backwards compat super often, and loads of people stick on LTS things, but you end up with packages that are just stuck without fixes of the “rename this import” variety.
This is part of why I do work with Jazzband (to just fix stuff), but I generally believe that our future will be much nicer if packages with any sort of real usage end up in some system that guarantees that we can find new maintainers if needed.
Maybe we need
python-contribthat just includes the things “most people use” (requests/attrsetc).This problem doesn’t go away from having it in the standard library. The maintainer goes away but the standard library as a whole still has maintainers. Bugs in that bit don’t get fixed because no one left remembers how it works. Eventually there’s a critical security vulnerability and the rest of the maintainers realise that the last person who knows how to fix it left the project a decade ago.
If anything, it’s more obvious in a separate project if there are still active maintainers.
I’m not familiar with the maintenance of python’s stdlib in particular, but in the general case I’ve noticed a stark difference. For example, in ruby or Go or JS, you’ll be far more likely to run into bugs or other problems with 3rd party modules. Corporate backing, the sheer number of people relying on it, the official identity of the language being tied to it…. these cultural forces seem to make a difference.
C and C++ have much smaller standard libraries but in both there are parts of common implementations of the standard library that very few people understand and that haven’t been touched for over 10 years. Mostly they work and so no one needs to touch the code but if there ever is a bug then good luck
Actually there is a core difference, in which the standard library has its own maintainers so if you personally want to start maintaining a thing that is not getting love, there are contact points that are, in theory, available and have the keys to the kingdom.
This is colored by my personal experience, but usually the lack of maintenance becomes a problem not from security issues, but from slow API changes in related libraries. Most people who have a habit of jumping into unknown code can look at a thing and try to untangle the mess in a implementation-compatible way. But if they don’t have release rights that work doesn’t go anywhere.
EDIT: There are exceptional cases to the contrary, but what I see a lot of is not projects without willing maintainers, but projects in limbo because passing the torch is difficult without a whole flow for determining that.
If you vendor it you start with this problem, and worse because someone may actually be maintaining their vendorerd copy and you won’t even know! Better to maintain a library people can use than everyone maintain their own copy. That’s why we invented these systems :)
PEP 594 does not plan to remove
optparse:Click is a good example of why it’s hard to remove: it’s a popular argument parsing library that was purposely built on top of
optparse.Click does not use optparse. It vendors a fork.
True in part (it doesn’t use the standard library version of
optparse, but it still uses a copy of it, specifically becauseoptparsewas already deprecated when Click was created), but the point still stands: there are projects out there that will need to do something similar, or they’ll break.Incredible that it still has tkinter, which in turn includes a full tcl interpreter dating long before the existence of Python itself…
Of course thirty years later, it’s not really “long before” anymore ;-) (only three years between Tcl and Python!)
tkinter could probably be moved outside of the standard library. Most distros package it separately anyway. I think the only reason it still gets bundled with Python is that it’d be nigh impossible to distribute from PyPI.
That, and also the fact that IDLE is a nice editor for teaching kids programming in Python, and I’d say a decent value-add in the ease of use sense (It comes with it’s own IDE!).
True! Though IDLE is desperately in need of some love to make it feel at least as friendly as VS Code. That doesn’t actually require much: maybe have Tkinter be less rubbish under MacOS, give a way to manage a project (even if it doesn’t understand version control, but for that, setuptools hooks exist), and maybe lean on jedi or and LSP daemon if it’s available?
it’s also super nice to have the
turtlemoduleThe cgi module! Such nostalgia seeing it go. I doubt 99% of the people have used it in the last decade or two. But I always thought of it as viable quick and dirty way to put together a webUI for all sorts of tasks to be executed on demand on a controlled environment. But in all fairness, utis has no use case I can think of that isn’t solved by http.server.
The tittle of this submission is a little misleading. “Finally”? Thid is a small set of long forgotten relics from past days. Perhaps accounting not more than dozens of users world wide.
I was kind of up in arms about this until I learned that the sources aren’t being removed. You’ll just need to vendor them to use them.
I DO understand this is additional work and nobody wants that, but I also think we all need to be a bit pragmatic about the fact that Python is HUGE and the people maintaining it only have so many hours in their lives :)
I’m glad the core team bit the bullet and is making this happen. I hope everyone affected can find their way to not sending too much vitriol upstream in response :)
Python still huge after removing this stuff.
I don’t honestly know what to do with a comment like this :)
Huge compared to what? C or Rust? Sure! Absolutely. But it’s meant to solve problems in a different domain.
Unless you’re talking MicroPython or CircuitPython, but then you’re just snarking so I guess it’s not up for actual discussion :)
I just meant: deprecating or removing parts of the standard library doesn’t seem like it has a big impact on the size/complexity. Still a big std lib, so is the breaking change worth it?
Every library that stays in needs ongoing maintenance in order to keep up with language changes and security compliance.
Also, with a toolset like Python which has enjoyed vast adoption, any change is a ‘breaking’ change for someone. The core team has to do the King Solomon thing and decide which course of action will benefit the most people.
Now, as much as I will ALWAYS have great fondness for Amiga IFF format libraries and uu-coding, these are tools that no longer serve the majority of the community’s needs.
So I think removing them from the standard distro but leaving them in Git so anyone who wants them can vendor them is a perfectly reasonable solution.
Woah, ossaudiodev was a thing in the standard library?! I had no idea about any of the audio related modules.