I’m a fan of pylint and recommend it to everyone. Is it much, much picker than other popular linters? Yes. But that’s a good thing, and typing “pylint” and an error/warning code into a search engine will bring up good documentation explaining what it was flagging and why, so for people newer to Python it’s a good educational tool as well. Here’s an example documentation page for one of the things flagged in the article.
And once you’ve worked with it for a while you start unconsciously writing “pylint style” – which is to say, better code than you used to – and it mostly fades into the background.
I’m a little confused why the author bemoans the pains of running pylint, but then dismisses pyflakes & flake8 as unnecessary, when they exist largely in response to those pains. Ruff may be the future, but if you’re not ready to make that jump, I’d be sticking with flake8.
Also given that pyenv wasn’t included in the sort of “prerequisite” basic tooling cited, I’d have included it. Some sort of dependency management would be a good fit, too… but that discussion could (and has) merited multiple articles of its own.
Huh, didn’t recognize the site, but I’d previously read that one wrt pyenv and seem to recall having some disagreements w/ some of the points in that one…might skim over it again though.
Imho, pyenv falls into the “fundamental building blocks that do one thing well” category of tools that belong in a dev’s python toolbox, at least at the moment. There may be some longer term solution that eliminates the need, but for now I wouldn’t leave it off the list.
FWIW I wholeheartedly recommend rtx as a replacement for pyenv, which you can use for the same purpose but for pretty much any runtime, not just Python, and it doesn’t rely on shims by default, but rather on updating PATH (unlike asdf).
I personally don’t particularly like the “integrate with everything” direction they’ve been going – I don’t want my version manager to be hooking into my shell, doing direnv stuff, spawning virtual environments, interacting w my package manager, etc… automatically, though I do understand the convenience for people who do.
Also, last I looked at asdf (and it’s been a while, and haven’t looked at all at what rtx has changed) it made it harder than pyenv if you needed to do anything custom in the cpython build… I’ve heard that’s improved, but haven’t checked again recently.
I agree regarding the extra stuff like creating/activating venvs, I only use it for managing runtimes and updating the PATH (which I guess does fall under “hooking into my shell,” though).
This gives a nice rundown of various failed sandboxing techniques for Python. The final line mentions that perhaps removing all double underscores would make it more or less impossible to circumvent the sandboxing. I tried looking around a bit to figure out a way through anyways, but haven’t found anything…
Well if you go into the article, you’ll see that you can isolate code from direct access to the builtins by passing in {"__builtins__": {}} into the global context when calling eval. So you can’t just type char, ord or getattr in there (eval("ord('a')", {"__builtins__": {}}) will fail because ord is not available in that context).
If you do have access to stuff like __class__ on attributes you can find you way either to code objects and the like, or outright to builtins (the main entry point is something like (1).__class__.__bases__[0].__subclasses__(), which gives you all subclasses of object currently loaded…).
But let’s say you scan the eval text for __ and disallow that! Is there another easy path to those builtins?
One can of course do "_" + "_" to get the double-underscored string. But without getattr you’re still kind of a bit stuck from what I’m seeing.
But let’s say you scan the eval text for __ and disallow that! Is there another easy path to those builtins?
You’d also need to disable any kind of recursive eval, because if you can write a program that generates text and eval that, then you have the same problems, and you can generate text by doing fun things like using double back-tick and then subtracting one from the character code in the text if it matches back tick, so anything searching for double underscores would fail.
I don’t think it was done “from an engineering perspective”. The obvious comparison is Svelte where there was a minor flap when they changed from .ts files to JSDoc files for various reasons. Rich Harris explained the reasons and life went on.
In this case, DHH doesn’t really provide any engineering justification for changing an existing project. If what it was doing before was wrong, why was it doing it wrong? For an internal tool, a “yolo” whatever attitude is fine, but in theory Turbo is a thing they want other developers to use, so normally you would try to get community input before making a decision with outsized impact on the community. It’s just a really weird way to behave with a project that has outside users.
Pep 723 looks like an amazing step forward. There’s still some trickiness because when you move from single-file scripts to something a bit bigger you might need to mess around a bit, but this feels like yet another step to making Python project setup have good options at every level of script granularity
Personally I think both PEPs 722 and 723 are pretty awful. There are basically two audiences here:
People who do know how to write out dependencies/package metadata already in the standard pyproject.toml file, but for whatever reason refuse to.
People who don’t know how to and also refuse to learn or do it.
People who do know how, and are willing to do it, and people who don’t know how but are willing to learn and do it, are already covered by the existing standard package-metadata file. So it’s just those two groups. And neither of them are really worth catering to; the first group will sooner or later move on to claiming that PEP 722/723 is too complicated and demand something simpler, and the second group will likely never adopt it in the first place, for similar reasons.
So it’s just pointless proliferation of “standards”.
I, a Python person, am above here, posting that it is interesting to me. I like it. There are plenty of people who like shipping things as single files. This is a nice little thing to help with that. Saying that I’m just going to whine about it and demand it will get changed because pyproject.toml is “complicated” is honestly extremely disagreeable to read.
Please consider that other people are actually reading what you’re writing.
I’ve been using Python for a decade and write plenty of packages. PEP 723 would really help me use Python for glue code, so I can have a single Python file in a non-Python project. I’ve got one ten-liner that’s called from a AutoHotKey shortcut, and it needs timeutils to work. Adding a pyproject.toml to a complex AHK codebase for one Python file is overkill.
The <script> tag you write to install htmx according to the docs looks like it is “static” but it’s not. Htmx has releases so that means it probably changes at least for security updates. So, I guess you could vendor it but it seems like what we used to do with jquery. Something came to fill a need after that even in jquery days (bower? i can’t remember).
So even if you don’t compile/transpile (build), you probably collect deps. So our CI pipelines and dev workflows probably look very similar.
Is there any testing library you could use that would also not have a build step? The backend probably has a testing library but that has a build step. So, you might have that already. I guess you could have an external dep (like a headless browser tester?) but that’s just a different class of dependency with a build step somewhere.
Not trying to FUD, I think htmx install is a slick intro. Just include a <script> tag but I think it’s different than Vue’s single page mode. If this turns into another “throw it in /static”, inline it in index.html, probably will not not great.
I started with Vue years ago espacialy because I could just include it with a tag.
That’s a huge bonus.
For a lot of my projects, this is enough. And even if now building is a breath of fresh air, thanks to vitejs, I still reach for it only if my project requires it.
Including things via a single CDN tag is an orthogonal issue to build steps. Alpine has a build step, but as a user you can just include the CDN tag or import the NPM module, your choice. Part of publishing a release is building the project and then pushing the built object up to NPM where it can be downloaded by the various JS CDNs.
I understand the desire for simplicity, but this is like saying, “I don’t want to own a car, so I live under a bridge.” You can live in a house and not own a car???
Including things with a script tag is orthogonal to hosting on a CDN. If you don’t have a build step, I can go to source, download the source file, then link to it as any other static files of my projects.
No, you can do that with eg Alpine too. It’s an unrelated concern to having a build step. Having a build step for the library does not impose a build step onto the consumer. You just distribute pre-built files. Even jQuery distributes a minified version.
I have nothing against vendoring with a package manager. Copying a file into /static and then thinking it’s done forever with no semver or basically thinking js/html/css is not code (ie: what the heck does static mean) that is real is the wrong mindset to me.
But whatever deals with change the best. If you have no change then you can do whatever (toy app, homework, prototype). No change happens, it doesn’t matter. Production? Declare all your deps (of all types). Don’t invent a package manager.
Even further, I think static buried in some pejorative folder path has strong sapir whorf.
Where did you programmatically get 1.9.2.js from? As a human I could go to the website, find latest is 1.9.2 and now I know the curl to type. But how does this work in the future? Are you going to have this script be called lock.sh? When you want to bump the version to 1.9.3, you update the version you want in lock.sh and then you invoke lock.sh and it downloads the version? Maybe it clears out all old files? That’s what I do with npm (or anything else) except I didn’t have to devise a system. What is the equivalent of npm outdated in CI if we are going to DIY a package manager? Or MD5 checksums beyond “the filename string says it is 1.9.2, it must be so”.
It’s not a big deal to update one piece of code, but the more code you have, the more important it is to have a single verified way to update everything. If you tried to install create-react-app by hand, you’d go crazy. I guess the idea with HTMX is you’re back in the JQuery days and you only have a small number of dependencies, say HTMX and your own JS. That’s fine, I guess, but I still prefer to use npm just for consistency and convenience.
Hopefully I don’t have to explain what the integrity attribute is doing.
Everything else–I would ask if that whole system is really needed? If so, you could always fall back to installing it with npm. But I am willing to bet that many people just don’t need that level of complexity.
When you copy it into /static, it loses the version number. This is how we used to do it with jQuery. I only knew the version because I would head the jQuery.min.js file. Then package managers came out and I generally don’t think this way. Everyone is free to do anything they want but I think this flow has many downsides.
Unpkg inline doesn’t work at a bank where I’m air-gapped and package scanners might not know what is going on. Copying the dep into /static is collecting deps but you are doing it manually. In this case, it’s just one dep but maybe that’s just lucky/rare. I mean, I’m all about minimizing deps, I just wonder what the app is. If you have tailwind, you have npm. Tailwind CDN is for “development purposes only”. So, I’m thinking more like tailwind is implying.
At a previous job (some years ago) I went completely nuts and wrote a service with Trio and wrote most functionality that was missing (e.g. a redis client). One reason was control and readability. With nurseries and cancellation (e.g. trio.move_on_after) the code was readable and could easily do timeouts and retries. I also noticed that I made much less mistakes in Trio than with asyncio.
In hindsight, someone should have stopped me. But I learned a lot!
I like the cooking metaphor for concurrency. You can start the water boiling before you go find the box of pasta. (Concurrency without parallelism.) Having two+ cooks will make things slower as you step on each other’s feet. (Lock contention.) Having a second kitchen doesn’t speed things up if you only have one cook. (Single threaded Python on a multicore machine.) Etc.
I don’t have a good kitchen metaphor for atomic operations yet though.
Atomic operations don’t map well to large-scale physical systems because most things at that scale are intrinsically atomic. Two people can’t reach the sink at once, so you effectively have mutual exclusion on washing up without needing atomicity. You might see atomicity violations at a serving hatch, where two people can reach for the same dish and one will get it the other will grab something else, for example.
The problem for the metaphor is that in the real world pretty much everything is atomic and thread safe as long as you look at what you’re doing before you do it. It’s hard to make examples of “you write down half a number and meanwhile someone else writes over the first half of your number but not the second” because people take up space and can see what they’re doing. The classic non-atomic bank transfer doesn’t happen with real money because you give away the dollar and then you don’t have a dollar to double spend.
Concurrency has a lot to do with sharing one resource, and Python has dedicated tools to deal with that depending on the resource you must share.
If you have to share one CPU while waiting on the network, then the specialized tools for this are asyncio, twisted, trio, gevent, etc.
Asyncio is the current standard to do this, but tornado, gevent and twisted solved this problem more than a decade ago. While trio and curio are showing us what the future could look like
It basically does everything the most basic UX designer teacher will tell you to avoid.
That was basically my reaction when we evaluated it for FreeBSD. We had advocates of each system configure a demo system for us to test. They were all implementing the same workflow. Everything in JIRA required between two and five clicks to do things that Bugzilla let us do in one. Bugzilla’s UI is pretty horrible, if you manage to build something worse then you must have been trying really hard.
pip is the default package installer that comes with Python.
poetry is a third-party package manager and package build tool with some workflow and workspace-management tools.
conda is a third-party package installer with a focus on the scientific/numeric community and on the broader ecosystem of not-just-Python tools and dependencies that community relies on.
I’m a bit confused why one of the examples suggest installing pipx or condax globally… wouldn’t you just install tools with pixi directly instead of pipx or condax?
That’s the whole problem with pixi and all the equivalent projects. The people that need them the most are the ones that are the least likely to understand how to use them correctly.
I’m pretty sure pixi will cause as many problems as it solves.
He I work at prefix, where did you get the feeling that we suggest using pipx or condax because that is not our intention. pixi global install PACKAGE installs a package that will then be globally accessible, like how pipx and condax works.
on a second reading, i can see this is saying “globally install tools in a manner similar to pipx or condax”, not “globally install tools such as pipx or condax”
The tenacity library comes with an intriguing mix of iterators and context managers to allow the users to retry blocks with this syntax:
from tenacity import Retrying, stop_after_attempt
for attempt in Retrying(stop=stop_after_attempt(3)):
with attempt:
1 / 0
You can use the same mechanism yourself, by creating and linking a custom context manager and iterator:
class YouAndIhaveUnfinishedBusiness:
def __init__(self, notify_success, attempt_number):
self.notify_success = notify_success
self.attempt = attempt_number
def __enter__(self):
pass
def __exit__(self, exc_type, exc_value, traceback):
if exc_value:
print(f"You have disappointed me {self.attempt} time(s).")
else:
self.notify_success()
return True
class DoOrDoNotThereIsNoTry:
def __init__(self, max_attempts):
self.max_attempts = max_attempts
self.success = False
def __iter__(self):
for i in range(self.max_attempts):
yield YouAndIhaveUnfinishedBusiness(self.succeed, i+1)
if self.success:
print("You are ready for the Kumite.")
break
else:
print("We trained him wrong on purpose, as a joke.")
def succeed(self):
self.success = True
Most of “Tech Twitter” seem to have migrated to various Mastodon instances, at least the corners that I frequented. There is an initial learning curve (or perhaps more accurately, un-learning from Twitter et al.) re: choosing an instance and platform differences, but I’ve been able to replicate my Twitter experience quite well on Mastodon so far. The quality of conversation is definitely better imo.
The problem is, 1% of quality posts from a giant instance will always be bigger, more diverse, and with higher chance of containing an amazing thing that 100% of whatever small instance you are on.
Plus now you have to do your homework: what is it federated with? What are the rules?
And what’s more, the recommendation system on mastodon is very limited, so you are basically stuck with what’s trending for everyone or your subscribe list.
Maybe you have a good instance to recommend for programming?
And what’s more, the recommendation system on mastodon is very limited, so you are basically stuck with what’s trending for everyone or your subscribe list.
You can follow hashtags in Mastodon. Click on one you like, then press the “follow” button, et voilà. Now you’ll see posts about this topic in your timeline.
Yep, this is a good point. You don’t get to wiggle out of the ‘doing your homework on the platforms you use’ step: large tech corporations just want you to skip the step and deal with the consequences later. Which is why Twitter is in this mess to begin with.
I’ve not personally found being on a smaller instance (social.coop) to be limiting in the slightest. Federation means that I think my experience is pretty much the same as I’d expect on a larger instance: I see all of the posts I care about from other instances, and I see posts from hashtags I follow.
I agree discoverability is probably Mastodon’s biggest hurdle at the moment, but given how ephemeral (in my opinion) social media is, and how easy it is to migrate your account, I haven’t worried too much about those “drawbacks” of federation - though it’s my understanding that your posts don’t move along with you, which to some may be not worth the hassle in the first place.
RE: instances, my main account is on .social, so I’m probably biased regarding discoverability, but most of my tech-oriented follows are split between .social, mastodon.design, and fosstodon.
The author leads with “People working in other industries should probably not be miserable at work either, but that is not the concern of this article”. About that:
I spent my 20s working almost-min-wage jobs in kitchens and grocery stores, working as many as 3 jobs (opening + prep work in a cafe in the early morning, cook in a restaurant in the afternoon and evening, and bus dishes on the weekend) and various side hustles to pay for a small room in a crowded house in South Berkeley (~approx 11 other people were living there), with not much hope in sight for anything different.
Sometimes nowadays I find myself getting frustrated with e.g. some of the nasty proprietary or legacy tech I have to work or interface with. But while this work can sometimes feel like slogging through filth, I’ve worked jobs where I literally had to slog my way through actual filth – and this is very far from that. As a knowledge worker, you generally have autonomy and respect and flexibility that is completely unheard of to even a lot of white collar workers, let alone folks toiling away doing physically demanding work for minimum wage. Not to mention you probably won’t deal with the kinds of psychological abuse and unsafe conditions that are part and parcel with many of those lower-wage jobs
Which isn’t to say that tech workers shouldn’t aim to improve their conditions further or try to have more fun at work or that we should put up with bullshit simply because others have it worse – It’s essential that we protect what we have and improve it and even enjoy ourselves. But I do think that tech workers often miss how dire things are for other folks, especially folks working low-wage, manual jobs, and it would be nice to see more recognition of the disparity in our circumstances
I grew up in a restaurant and spent some time working as a bus boy. It really grinds my gears when you go out for a meal with a coworker and they complain about the service. “How hard could it be to get my order right?” Why don’t you work in a restaurant for a couple years and find out! Or when people assume scaling a restaurant is as easy as adding a load balancer and a couple more servers (pun not intended, but appreciated).
Some people have never worked in the service industry and it really shows.
I resonate really hard with this, I’ve come back several times to try to write a reply that isn’t a whole rant about people in tech but:
I’ve done a bunch of not so sexy jobs (legal and not so much, I’ll leave those out): retail, restaurants in various positions, and I was even one of those street fundraisers for the children (where I was subject to physical violence and the company’s reaction was “it comes with the job”). Now I work tech full time, and I’m a deckhand when I’m not doing that.
My perspective is shaped by a couple things, I think:
Being born to teenage parents who worked similar jobs and struggled for a long time
The fact that they “raised me right” – if I talked to / treated anyone the way I’ve seen some folks I’ve met in this industry do to service workers / people they seem to consider as “below them”, they wouldn’t be having any of it
Actually working the jobs and being subject to the awful treatment by both customers and management
The thing is, though, is that I really don’t think you should have to go through any of this in order to just not be a jerk to people…I really don’t know what the disconnect is. The most difficult customers I’ve had (at previous jobs and on the boat) have typically been the ones that seem the most privileged. When it comes to restaurants, the cheapest ones (in terms of tipping) were similarly the people that would come in and order hundreds of dollars worth of food and then leave little to no tip (I’m not here to debate tipping culture, it is what it is here at this point).
I’ve had situations where I take people to a place where I’m cool with the staff and someone picks up the tab (for which I’m appreciative) but then they are rude / pushy / skimp out on the tip, which is really embarrassing to say the least (I don’t hesitate to call people out but I feel like … I shouldn’t have to?)
The boat I work on is in the bay area and so we get a lot of tech people, and a couple of things stand out:
I don’t really know how some of the most intelligent people can be so dumb (literally all you need to do is follow directions)
They talk down to us (the crew trying to put them on fish and, for what it’s worth, keep them alive – won’t get into that here), and when you ask them not to do something for safety or you try to correct something they’re doing wrong, they get an attitude. I want to emphasize, not everyone, but enough to make you stop and ask why.
When they find out that I also work in tech (you talk to people since you’re with them for 8+ hours), the reaction is typically one of “why do you need to be doing THIS?”. Sidenote – the most hilarious thing that I enjoy doing is dropping into a technical conversation (a lot of people come on with their coworkers) and having people be like “wtf how does the deckhand know about any of this?”
They don’t tip … lol … or they stiff us for fish cleaning which we are upfront is a secondary service provided for a fee.
It’s not everyone, but I get a pretty decent sample size given the population here. The plus side of working on the boat (vs a restaurant or retail) is that if someone starts being a major a-hole the captain doesn’t mind if we stand up for ourselves (encourages it, even)
It’s not everyone of course, but it’s enough to make you wonder.
Some people have never worked in the service industry and it really shows.
Yeah, exactly. Or, we have a saying “you can tell whose never pushed a broom in their life”.
That was more of a rant than I wanted to get into but since I’m here it was kind of cathartic. I really just wish people would stop and think about the human on the other end. Of course it’s not just tech people that do things like this, but … yeah.
I’ve done a bunch of not so sexy jobs (legal and not so much, I’ll leave those out)
I worked in eDiscovery for a while (~3 years) so I have a sense of legal. It’s very stratified and stressful. I remember going to bed at 2 AM and waking up at 5 AM to make sure that a production was ready for opposing counsel. Not ideal…
My perspective is shaped by a couple things, I think:
Being born to teenage parents who worked similar jobs and struggled for a long time
By contrast, my father was 39 when he had me. However, he had a hard life. He grew up in Francoist Spain. (One of the few memories of my grandfather was when he told me “La habre es miseria. La habre es miseria.” (Hunger is misery. Hunger is misery.)) My father was a Spanish refuge in France at age 9. He didn’t complete high school. Instead, he did an apprenticeship in a French restaurant where the chefs beat him. He worked 16 hour days for a long time.
The fact that they “raised me right” – if I talked to / treated anyone the way I’ve seen some folks I’ve met in this
industry do to service workers / people they seem to consider as “below them”, they wouldn’t be having any of it
Absolutely. My father always said that everyone was welcome at his restaurant, regardless of what they were wearing. It’s important to respect everyone.
Inter-generational trauma is a real thing. I’m doing okay, but my brothers didn’t fare so well. (A topic for a more one on one conversation.) I hope you are okay. <3
Edit: this has really thrown me through a loop. I don’t mean to be dramatic and I know this is a public forum, but I’m sure there are more people posting than responding. If it means anything to anyone then it’s more important to say so than to be stoic. I hope you are all doing okay.
I worked in eDiscovery for a while (~3 years) so I have a sense of legal. It’s very stratified and stressful. I remember going to bed at 2 AM and waking up at 5 AM to make sure that a production was ready for opposing counsel. Not ideal…
Heh, sorry I meant legal vs not-so-legal in the legality sense, but in any case wow that sounds dreadful!
I appreciate your kind words and you sharing your story, and I’m glad you’re doing okay. I’m also sorry to hear about your brothers, similar thing is true for some of my siblings…kind of weird how that works.
There is a theory in some circles that states that having money enable people to not have to care about other human beings. With money you can feel like you provide for all your needs by just buying stuff and services. If you don’t have so much money, you need to compensate by trying to build mutual understanding. That leads to being more empathic. You also need to respond or even anticipate the needs of people who give you money. Which leads also to some kind of asymmetric empathy (similar to impostor syndrome).
Also there may be the fact that some people are attracted to tech because they fell they are more gifted with machines than with people. So maybe some form of selection bias here too.
Well put. I sometimes ask myself, “How many people are living miserable lives so that I can sit in a cushy chair and think about interesting problems?”
Well, I’ve worked in the oil and gas industry, so I helped keep lots of people’s heat working in the winter, including my own. At the cost of making the world incrementally more fucked though, so that one’s a net negative. I’ve done a fair amount of teaching, so I helped share skills that were useful for people. I’ve worked datacenter tech support, so I helped lots of people keep their online businesses working. So there’s that.
If I really wanted to make the world a better place, I would either work for something like Habitat For Humanity and build houses, or I would get a PhD in nuclear physics or material science and try to make better nuclear/solar energy sources. Or become a teacher, natch, but my sister and both parents are teachers so I feel like I have to break the family mold. Could always do what my grandmother did and become a social worker. Or go into politics, because someone with a brain stem has to, but I’ve had enough brushes with administration to know that I will burn myself out and be of no use to anyone.
Right now I work in robotics doing R&D for autonomous drones, so I’ll hopefully make peoples’ lives better somewhere, someday down the line. Nothing as good as what Zipline does, but on a similar axis – one of my most fun projects was basically Zipline except for the US Army, so it was 10x more expensive and 0.25x as good.
…do people not normally think about this sort of thing?
That there are people supporting cole-k’s job (I don’t know who, maybe car mechanics, cafeteria workers, janitors?) whose work is required for cole-k’s job to be possible, but who are necessarily miserable in their jobs.
Yeah, and I’ve done at least a moderate share of supporting them back one way or another, within the bounds of my aptitudes, skills, and how selfish I’m willing to be – and honestly I’m pretty selfish, ngl. Sometimes I’ve done it directly by serving them back, more often indirectly by trying to do things that Benefit Everyone A Bit. All I can do is keep trying. We’re all in this together.
This should not be controversial, and sometimes I wish I had a button to teleport some of my colleagues where I used to work in Africa to recalibrate their sense of what “hard” means.
This is so true. I try to remind myself of this as much as I can but as I did not experience minimum wage work myself this can be hard to be fully aware of this situation.
Maybe we should try for the condition of everybody to improve. I fear that by insisting a lot on the good conditions we have in the tech industry it would only encourage a degradation on those conditions unfortunately. Tactically, I wonder if we should not focus on the commonalities of the issues we face across all the different types of jobs.
I also paid for college and university working in a large hotel kitchen and then dining room. At the time in the front of the house I could earn enough in tips over a summer to cover a year of state school tuition plus room and board. I’d go back on holidays and long weekends to make the rest of my expenses. It was hard work, long hours, and disrespected in all kinds of ways. Once in a while there was violence or the threat of violence. But it beat doing landscaping or oil changes and tires. There were a number of us who were using it as a stepping stone, one guy from Colombia worked until he saved up enough to go back home and buy a multi-unit building so he and his parents could live in it and be landlords, get a used BMW for himself, and finish his education. His motivation, and taking extra shifts, made mine look weak and I was highly motivated.
I remind myself of that time when I’m frustrated at my desk.
one guy from Colombia worked until he saved up enough to go back home and buy a multi-unit building so he and his parents could live in it and be landlords
This I think was one of Bryan Caplan’s arguments about open borders.
In addition to the moral issue that no-one has the right to curtail the free movement of others[1], there is solid empirical evidence that not only do immigrants enrich the countries they emigrate to (i.e. contribute more on average than locals), they often also help lift their home countries out of poverty by doing exactly what your Colombian friend did.
[1] One frequently occurring example of hypocrisy on the matter of travel: people who simultaneously rail against any attempt by their own Government to control their movement (passports, papers, ID, etc.), but also complain loudly about people crossing the border into “their” country and demand the Government build a wall, metaphorically or literally.
I’ve encountered this use case, it would be handy, but adding magic to comments is always dangerous. I would rather include a requirements.txt and give a way to have “ensure these requirements and then run” in a shebang
In the meantime… you can run pip directly at the start of the script, run pip install in a directory, node_modules style, change sys.path and continue with the script.
Yeah, the use case of “I want to write a script and distribute it, but not involve any proper packaging” should really just consist of handing someone a zip file with the script and some instructions on how to set it up, rather than trying to get the packaging ecosystem to support it.
PEX by default builds a (better) zipapp. The resulting executable is more portable than a regular zipapp because it bundles an interpreter with versioned dependencies and executes it in an isolated environment. I would literally never choose to use a zipapp over a pex file.
I read that comment. And I don’t think the “but what about C extensions” bit is relevant. Somebody who has a single-file script with a C extension that they want to distribute to others without properly packaging it… is someone I’m happy to leave unsupported. Packaging is easy enough at this point that for any code that’s even moderately complex (and a C extension is way past that point) it should just be the default. And for the single-file script whose author refuses to use proper packaging tools, zipapp ought to be the way to go. Which leaves no space for this proposal.
beginners, students, kids, that just do small things and installed pillow or pygame and wanted to share that with their friends but barely know the ecosystem
sysadmin that want a quick and dirty script a thousands time a day to check something out on machine x that happens need a compiled driver to access z and they work on windows but test on linux
thousands of professionals that are not coders, like biologist, mathematicians, geographers and analysts that just know a little bit of python and have neither the time, desire or skill to learn an entire ecosystem to make share their matplot lib graphs with their colleagues by email
This reads like somebody living in the bubble of pro dev, forgetting a gigantic part of the Python community is not. That’s the appeal of the language. And we should cater to them to. They suffer enough with packaging.
And as a professional Python dev that started in 2.4, I would be very happy with having a way to add things like fabric with on line in my numerous scripts that I touch once a year. I don’t want to have requirements files everywhere for 50 modules of 20 lines that have been created over 15 years and have no congruency whatsoever.
I would pay real money to never again have someone bust out the “if you don’t support this exact proposal, then you must not care about beginners and students!” fallacy ever again.
The zipapp technique already exists, and it works for this use case (“I have a script, I don’t want to learn packaging, just let me send it to someone else”). You yourself admit that it works for this use case. The case where it doesn’t work is the case of also wanting to include a custom compiled C extension with the script, and if you think beginning Python programmers are routinely doing that, then I’d be interested to know exactly who your “beginners” are.
And let’s face it: if the complaint is that, say, learning to make a requirements file in order to reproduce an environment is too difficult and complex and too much effort, then how on earth is “let’s make them learn to make a requirements file, we’ll just store it in a different place” an acceptable alternative?
It seems that for some reason you like this proposal. But you need to A) be honest about the alternatives, B) be honest about its flaws and C) understand that other people may also be aware of the issues faced by beginners, students, academics, etc. but may reasonably come to different conclusions about how best to handle those use cases.
Thanks to c-extensions, it’s not cross platform. And they require your source to be zipped, which is less than ideal if you edit it often.
This proposal has several benefits:
it’s simple enough beginners can understand and use it
it’s clear enough people will be able to share around it. zipapp are not clear
it works cross platform
it doubles as documentation
it solves the “I have many small scripts with different requirements in the same dir” problem, which is common for the part of the community doing data analysis, finance, biology and sysadmin. A.K.A pros coding in python, but not pro python devs
Personally, I like it a lot.
When I’m in a project, I don’t care.
But at home, I have a Scripts dirs with a lot of small quick and dirty 20 utility utilities. I have to avoid using requests, keyring or typer because dealing with deps on those would be annoying. This would solve that.
Well, I mostly switched from Python to Ruby (with the inline Bundler mode, which this PEP tries to replicate – badly, in my opinion) for scripts not because I haven’t heard of zipapp (actually, I haven’t, thank you; I heard of PEX, though), but because I don’t want the extra hassle of dealing with dependency management outside the script. Zipapp solves the problem by requiring me to package my scripts into archives. You highlight this and other problems yourself.
If I need to run an extra step to run each of my scripts, why not use a compiled language like Nim then? That way, once my script compiles, it will always work without a fault from /usr/local/bin. Which cannot be said about Python/Ruby scripts when system packages upgrade.
At the same time, the inline Bundler mode in Ruby actually reduces the burden of dependency management by ensuring that when I run ruby script.rb, I have one less thing to worry about. And, as opposed to this PEP, inline Bundler config is (a) real Ruby code, and (b) uses the same syntax in the inline and Gemfile (standalone) modes.
Good tips. This probably isn’t as relevant to the author’s target audience, but it used to be that to use
pdbyou had to doimport pdb; pdb.set_trace()But as of 2017 you no longer have to explicitly import pdb, you can just call the built-in
breakpoint. So just dobreakpoint()instead. Mentioning because a surprising number of experienced Python users don’t seem to know about even pdb
python -m pdb script.pyrunsscriptuntil it throws an exception, then drops into pdb.Yes, I link to it in the article, but there is a dedicated article on how to use pdb that mentions breakpoint(): https://www.bitecode.dev/p/intro-to-pdb-the-python-debugger
I will likely write a similar article for pylint, mypy, etc. But they are not there yet.
I’m a fan of
pylintand recommend it to everyone. Is it much, much picker than other popular linters? Yes. But that’s a good thing, and typing “pylint” and an error/warning code into a search engine will bring up good documentation explaining what it was flagging and why, so for people newer to Python it’s a good educational tool as well. Here’s an example documentation page for one of the things flagged in the article.And once you’ve worked with it for a while you start unconsciously writing “pylint style” – which is to say, better code than you used to – and it mostly fades into the background.
Yes, pylint is the best linter precisely because it’s so agressive. Still, I never use the default settings, its just too much for me.
Seems like mostly good advice.
I’m a little confused why the author bemoans the pains of running pylint, but then dismisses pyflakes & flake8 as unnecessary, when they exist largely in response to those pains. Ruff may be the future, but if you’re not ready to make that jump, I’d be sticking with flake8.
Also given that pyenv wasn’t included in the sort of “prerequisite” basic tooling cited, I’d have included it. Some sort of dependency management would be a good fit, too… but that discussion could (and has) merited multiple articles of its own.
The blog has a bit on deps management: “Reliving packaging pain” (https://www.bitecode.dev/p/relieving-your-python-packaging-pain)
It also has a bit on why it will not recommend pyenv: “Why not tell people to “simply” use pyenv, poetry or anaconda” (https://www.bitecode.dev/p/why-not-tell-people-to-simply-use)
Huh, didn’t recognize the site, but I’d previously read that one wrt pyenv and seem to recall having some disagreements w/ some of the points in that one…might skim over it again though.
Imho, pyenv falls into the “fundamental building blocks that do one thing well” category of tools that belong in a dev’s python toolbox, at least at the moment. There may be some longer term solution that eliminates the need, but for now I wouldn’t leave it off the list.
FWIW I wholeheartedly recommend rtx as a replacement for pyenv, which you can use for the same purpose but for pretty much any runtime, not just Python, and it doesn’t rely on shims by default, but rather on updating PATH (unlike asdf).
I personally don’t particularly like the “integrate with everything” direction they’ve been going – I don’t want my version manager to be hooking into my shell, doing direnv stuff, spawning virtual environments, interacting w my package manager, etc… automatically, though I do understand the convenience for people who do.
Also, last I looked at asdf (and it’s been a while, and haven’t looked at all at what rtx has changed) it made it harder than pyenv if you needed to do anything custom in the cpython build… I’ve heard that’s improved, but haven’t checked again recently.
I agree regarding the extra stuff like creating/activating venvs, I only use it for managing runtimes and updating the PATH (which I guess does fall under “hooking into my shell,” though).
This gives a nice rundown of various failed sandboxing techniques for Python. The final line mentions that perhaps removing all double underscores would make it more or less impossible to circumvent the sandboxing. I tried looking around a bit to figure out a way through anyways, but haven’t found anything…
You can’t do that either because of char, ord, getattr and miriads of other technics to introspect objects in python.
Well if you go into the article, you’ll see that you can isolate code from direct access to the builtins by passing in
{"__builtins__": {}}into the global context when callingeval. So you can’t just typechar,ordorgetattrin there (eval("ord('a')", {"__builtins__": {}})will fail becauseordis not available in that context).If you do have access to stuff like
__class__on attributes you can find you way either tocodeobjects and the like, or outright tobuiltins(the main entry point is something like(1).__class__.__bases__[0].__subclasses__(), which gives you all subclasses ofobjectcurrently loaded…).But let’s say you scan the
evaltext for__and disallow that! Is there another easy path to those builtins?One can of course do
"_" + "_"to get the double-underscored string. But withoutgetattryou’re still kind of a bit stuck from what I’m seeing.You’d also need to disable any kind of recursive eval, because if you can write a program that generates text and eval that, then you have the same problems, and you can generate text by doing fun things like using double back-tick and then subtracting one from the character code in the text if it matches back tick, so anything searching for double underscores would fail.
At least one, but I feel like helping people patch up sandboxes based on eval + string replacement might not be a responsible thing to do
This is purely a game for me, I am curious is all. ast.literal_eval is sitting there for “real” use cases anyways
If you are able to find a way out of this sandbox, please tag me to tell me about it because this is relevant to my interests.
Thanks!
Without __, you don’t even have OOP. And without even builtins, Python is a glorified calculator. What would be the point?
You can sandbox anything down to bitshift instructions, for sure. The hard part is making it useful.
So many disproportionate reactions to this announce.
That’s a lot of emotion for what basically boils down to something you can evaluate from an engineering perspective.
I don’t think it was done “from an engineering perspective”. The obvious comparison is Svelte where there was a minor flap when they changed from .ts files to JSDoc files for various reasons. Rich Harris explained the reasons and life went on.
In this case, DHH doesn’t really provide any engineering justification for changing an existing project. If what it was doing before was wrong, why was it doing it wrong? For an internal tool, a “yolo” whatever attitude is fine, but in theory Turbo is a thing they want other developers to use, so normally you would try to get community input before making a decision with outsized impact on the community. It’s just a really weird way to behave with a project that has outside users.
That you have little information and you disagree with the decision doesn’t really change my initial point.
Pep 723 looks like an amazing step forward. There’s still some trickiness because when you move from single-file scripts to something a bit bigger you might need to mess around a bit, but this feels like yet another step to making Python project setup have good options at every level of script granularity
Personally I think both PEPs 722 and 723 are pretty awful. There are basically two audiences here:
pyproject.tomlfile, but for whatever reason refuse to.People who do know how, and are willing to do it, and people who don’t know how but are willing to learn and do it, are already covered by the existing standard package-metadata file. So it’s just those two groups. And neither of them are really worth catering to; the first group will sooner or later move on to claiming that PEP 722/723 is too complicated and demand something simpler, and the second group will likely never adopt it in the first place, for similar reasons.
So it’s just pointless proliferation of “standards”.
I, a Python person, am above here, posting that it is interesting to me. I like it. There are plenty of people who like shipping things as single files. This is a nice little thing to help with that. Saying that I’m just going to whine about it and demand it will get changed because
pyproject.tomlis “complicated” is honestly extremely disagreeable to read.Please consider that other people are actually reading what you’re writing.
I’m a python expert with 20 years of experience in it. I packaged a lot, even before pip and pyproject were a thing.
I love the proposal, as I have lots of small scripts and writing a config file for each of them is annoying so I stick to stdlib.
I’ve been using Python for a decade and write plenty of packages. PEP 723 would really help me use Python for glue code, so I can have a single Python file in a non-Python project. I’ve got one ten-liner that’s called from a AutoHotKey shortcut, and it needs
timeutilsto work. Adding apyproject.tomlto a complex AHK codebase for one Python file is overkill.The
<script>tag you write to install htmx according to the docs looks like it is “static” but it’s not. Htmx has releases so that means it probably changes at least for security updates. So, I guess you could vendor it but it seems like what we used to do with jquery. Something came to fill a need after that even in jquery days (bower? i can’t remember).So even if you don’t compile/transpile (build), you probably collect deps. So our CI pipelines and dev workflows probably look very similar.
Is there any testing library you could use that would also not have a build step? The backend probably has a testing library but that has a build step. So, you might have that already. I guess you could have an external dep (like a headless browser tester?) but that’s just a different class of dependency with a build step somewhere.
Not trying to FUD, I think htmx install is a slick intro. Just include a
<script>tag but I think it’s different than Vue’s single page mode. If this turns into another “throw it in /static”, inline it inindex.html, probably will not not great.I started with Vue years ago espacialy because I could just include it with a tag.
That’s a huge bonus.
For a lot of my projects, this is enough. And even if now building is a breath of fresh air, thanks to vitejs, I still reach for it only if my project requires it.
Including things via a single CDN tag is an orthogonal issue to build steps. Alpine has a build step, but as a user you can just include the CDN tag or import the NPM module, your choice. Part of publishing a release is building the project and then pushing the built object up to NPM where it can be downloaded by the various JS CDNs.
I understand the desire for simplicity, but this is like saying, “I don’t want to own a car, so I live under a bridge.” You can live in a house and not own a car???
Including things with a script tag is orthogonal to hosting on a CDN. If you don’t have a build step, I can go to source, download the source file, then link to it as any other static files of my projects.
No, you can do that with eg Alpine too. It’s an unrelated concern to having a build step. Having a build step for the library does not impose a build step onto the consumer. You just distribute pre-built files. Even jQuery distributes a minified version.
I don’t think I understand. Vendoring is bad because people used to vendor jquery?
I have nothing against vendoring with a package manager. Copying a file into
/staticand then thinking it’s done forever with no semver or basically thinking js/html/css is not code (ie: what the heck does static mean) that is real is the wrong mindset to me.But whatever deals with change the best. If you have no change then you can do whatever (toy app, homework, prototype). No change happens, it doesn’t matter. Production? Declare all your deps (of all types). Don’t invent a package manager.
Even further, I think
staticburied in some pejorative folder path has strong sapir whorf.curl ... -o static/htmx.1.9.2.jsAnd now I have a versioned local static script.
Everything doesn’t have to be difficult.
Where did you programmatically get
1.9.2.jsfrom? As a human I could go to the website, find latest is1.9.2and now I know thecurlto type. But how does this work in the future? Are you going to have this script be calledlock.sh? When you want to bump the version to1.9.3, you update the version you want inlock.shand then you invokelock.shand it downloads the version? Maybe it clears out all old files? That’s what I do with npm (or anything else) except I didn’t have to devise a system. What is the equivalent ofnpm outdatedin CI if we are going to DIY a package manager? Or MD5 checksums beyond “the filename string says it is 1.9.2, it must be so”.It’s not a big deal to update one piece of code, but the more code you have, the more important it is to have a single verified way to update everything. If you tried to install create-react-app by hand, you’d go crazy. I guess the idea with HTMX is you’re back in the JQuery days and you only have a small number of dependencies, say HTMX and your own JS. That’s fine, I guess, but I still prefer to use npm just for consistency and convenience.
I don’t get where people got “never use npm” from the original post. You can totally npm install htmx.org and serve it from the node_modules directory
That was just an example off the top of my head, in practice you go to the install instructions and follow that: https://htmx.org/docs/#via-a-cdn-e-g-unpkg-com
The install instructions answer this question:
Hopefully I don’t have to explain what the
integrityattribute is doing.Everything else–I would ask if that whole system is really needed? If so, you could always fall back to installing it with npm. But I am willing to bet that many people just don’t need that level of complexity.
What do you mean by collect deps? You do load HTMX like jQuery, just put htmx.js in /static or use the unpkg mirror and roll with it.
When you copy it into /static, it loses the version number. This is how we used to do it with jQuery. I only knew the version because I would
headthejQuery.min.jsfile. Then package managers came out and I generally don’t think this way. Everyone is free to do anything they want but I think this flow has many downsides.Unpkg inline doesn’t work at a bank where I’m air-gapped and package scanners might not know what is going on. Copying the dep into /static is collecting deps but you are doing it manually. In this case, it’s just one dep but maybe that’s just lucky/rare. I mean, I’m all about minimizing deps, I just wonder what the app is. If you have tailwind, you have
npm. Tailwind CDN is for “development purposes only”. So, I’m thinking more like tailwind is implying.Just add the version number in the name. You will upgrade it once every 3 years, it’s a 20 seconds job. Still much simpler.
At a previous job (some years ago) I went completely nuts and wrote a service with Trio and wrote most functionality that was missing (e.g. a redis client). One reason was control and readability. With nurseries and cancellation (e.g.
trio.move_on_after) the code was readable and could easily do timeouts and retries. I also noticed that I made much less mistakes in Trio than with asyncio.In hindsight, someone should have stopped me. But I learned a lot!
There is no question that trio is a superior async library, that gives you better results all being equal.
But yeah, it’s rarely worth it.
I like the cooking metaphor for concurrency. You can start the water boiling before you go find the box of pasta. (Concurrency without parallelism.) Having two+ cooks will make things slower as you step on each other’s feet. (Lock contention.) Having a second kitchen doesn’t speed things up if you only have one cook. (Single threaded Python on a multicore machine.) Etc.
I don’t have a good kitchen metaphor for atomic operations yet though.
Tasting then adding the salt is not atomic.
If you taste, then the door rings, the you come back and add the salt, someone might have done the same while you were away.
Atomic operations don’t map well to large-scale physical systems because most things at that scale are intrinsically atomic. Two people can’t reach the sink at once, so you effectively have mutual exclusion on washing up without needing atomicity. You might see atomicity violations at a serving hatch, where two people can reach for the same dish and one will get it the other will grab something else, for example.
Atomic operations are operations that are so simple that acquiring the means to do them coincides with doing them.
Opening a tin or a lid: an item with closed lid is consumed to yields an item with open lid.
Stirring a pot: an unstirred pot is turned into a stirred pot. Stirring is an irreversible action.
Peeling a potato. Squeezing a lemon. All atomic.
The problem for the metaphor is that in the real world pretty much everything is atomic and thread safe as long as you look at what you’re doing before you do it. It’s hard to make examples of “you write down half a number and meanwhile someone else writes over the first half of your number but not the second” because people take up space and can see what they’re doing. The classic non-atomic bank transfer doesn’t happen with real money because you give away the dollar and then you don’t have a dollar to double spend.
Summary:
Concurrency has a lot to do with sharing one resource, and Python has dedicated tools to deal with that depending on the resource you must share.
If you have to share one CPU while waiting on the network, then the specialized tools for this are asyncio, twisted, trio, gevent, etc.
Asyncio is the current standard to do this, but tornado, gevent and twisted solved this problem more than a decade ago. While trio and curio are showing us what the future could look like
But chances are, you should use none of them.
I think the comments all agree: yes we hate JIRA.
For me it’s the terrible UI:
It basically does everything the most basic UX designer teacher will tell you to avoid.
That was basically my reaction when we evaluated it for FreeBSD. We had advocates of each system configure a demo system for us to test. They were all implementing the same workflow. Everything in JIRA required between two and five clicks to do things that Bugzilla let us do in one. Bugzilla’s UI is pretty horrible, if you manage to build something worse then you must have been trying really hard.
I’m not a Python dev - what’s the difference between “conda” and “pip” or “poetry”? I’ve only used pip/poetry
pipis the default package installer that comes with Python.poetryis a third-party package manager and package build tool with some workflow and workspace-management tools.condais a third-party package installer with a focus on the scientific/numeric community and on the broader ecosystem of not-just-Python tools and dependencies that community relies on.Also, conda is tied to anaconda, which is a commercial product, albeit with a free ofter.
But I would advice using anything else than pip if you have to ask this question.
See: https://www.bitecode.dev/p/relieving-your-python-packaging-pain and https://www.bitecode.dev/p/why-not-tell-people-to-simply-use
I’m a bit confused why one of the examples suggest installing pipx or condax globally… wouldn’t you just install tools with pixi directly instead of pipx or condax?
That’s the whole problem with pixi and all the equivalent projects. The people that need them the most are the ones that are the least likely to understand how to use them correctly.
I’m pretty sure pixi will cause as many problems as it solves.
I’m probably borderline spamming at this point, but better post a link again than explain the whole thing in a comment repeatedly: https://www.bitecode.dev/p/why-not-tell-people-to-simply-use
He I work at prefix, where did you get the feeling that we suggest using
pipxorcondaxbecause that is not our intention.pixi global install PACKAGEinstalls a package that will then be globally accessible, like howpipxandcondaxworks.from the article:
on a second reading, i can see this is saying “globally install tools in a manner similar to pipx or condax”, not “globally install tools such as pipx or condax”
This needs to be rewritten to be less ambiguous.
Summary:
The tenacity library comes with an intriguing mix of iterators and context managers to allow the users to retry blocks with this syntax:
You can use the same mechanism yourself, by creating and linking a custom context manager and iterator:
Summary:
Python without the GIL, for good
LPython: a new Python Compiler
Pydantic 2 is getting usable
PEP 387 defines “Soft Deprecation”, getopt and optparse soft deprecated
Cython 3.0 released with better pure Python support
PEP 722 – Dependency specification for single-file scripts
Python VSCode support gets faster
Paint in the terminal
Most of “Tech Twitter” seem to have migrated to various Mastodon instances, at least the corners that I frequented. There is an initial learning curve (or perhaps more accurately, un-learning from Twitter et al.) re: choosing an instance and platform differences, but I’ve been able to replicate my Twitter experience quite well on Mastodon so far. The quality of conversation is definitely better imo.
The problem is, 1% of quality posts from a giant instance will always be bigger, more diverse, and with higher chance of containing an amazing thing that 100% of whatever small instance you are on.
Plus now you have to do your homework: what is it federated with? What are the rules?
And what’s more, the recommendation system on mastodon is very limited, so you are basically stuck with what’s trending for everyone or your subscribe list.
Maybe you have a good instance to recommend for programming?
You can follow hashtags in Mastodon. Click on one you like, then press the “follow” button, et voilà. Now you’ll see posts about this topic in your timeline.
It would have benefitted us to do this homework before we ever engaged with Facebook, Twitter, or reddit, before they all exploded in production.
Federation is the static type-system of social networking :^)
Yep, this is a good point. You don’t get to wiggle out of the ‘doing your homework on the platforms you use’ step: large tech corporations just want you to skip the step and deal with the consequences later. Which is why Twitter is in this mess to begin with.
I’ve not personally found being on a smaller instance (social.coop) to be limiting in the slightest. Federation means that I think my experience is pretty much the same as I’d expect on a larger instance: I see all of the posts I care about from other instances, and I see posts from hashtags I follow.
“Programming” is a little too general I think, but for more specialized topics functional.cafe and mastodon.gamedev.place both seem solid.
I agree discoverability is probably Mastodon’s biggest hurdle at the moment, but given how ephemeral (in my opinion) social media is, and how easy it is to migrate your account, I haven’t worried too much about those “drawbacks” of federation - though it’s my understanding that your posts don’t move along with you, which to some may be not worth the hassle in the first place.
RE: instances, my main account is on .social, so I’m probably biased regarding discoverability, but most of my tech-oriented follows are split between .social, mastodon.design, and fosstodon.
The author leads with “People working in other industries should probably not be miserable at work either, but that is not the concern of this article”. About that:
I spent my 20s working almost-min-wage jobs in kitchens and grocery stores, working as many as 3 jobs (opening + prep work in a cafe in the early morning, cook in a restaurant in the afternoon and evening, and bus dishes on the weekend) and various side hustles to pay for a small room in a crowded house in South Berkeley (~approx 11 other people were living there), with not much hope in sight for anything different.
Sometimes nowadays I find myself getting frustrated with e.g. some of the nasty proprietary or legacy tech I have to work or interface with. But while this work can sometimes feel like slogging through filth, I’ve worked jobs where I literally had to slog my way through actual filth – and this is very far from that. As a knowledge worker, you generally have autonomy and respect and flexibility that is completely unheard of to even a lot of white collar workers, let alone folks toiling away doing physically demanding work for minimum wage. Not to mention you probably won’t deal with the kinds of psychological abuse and unsafe conditions that are part and parcel with many of those lower-wage jobs
Which isn’t to say that tech workers shouldn’t aim to improve their conditions further or try to have more fun at work or that we should put up with bullshit simply because others have it worse – It’s essential that we protect what we have and improve it and even enjoy ourselves. But I do think that tech workers often miss how dire things are for other folks, especially folks working low-wage, manual jobs, and it would be nice to see more recognition of the disparity in our circumstances
I grew up in a restaurant and spent some time working as a bus boy. It really grinds my gears when you go out for a meal with a coworker and they complain about the service. “How hard could it be to get my order right?” Why don’t you work in a restaurant for a couple years and find out! Or when people assume scaling a restaurant is as easy as adding a load balancer and a couple more servers (pun not intended, but appreciated).
Some people have never worked in the service industry and it really shows.
I resonate really hard with this, I’ve come back several times to try to write a reply that isn’t a whole rant about people in tech but:
I’ve done a bunch of not so sexy jobs (legal and not so much, I’ll leave those out): retail, restaurants in various positions, and I was even one of those street fundraisers for the children (where I was subject to physical violence and the company’s reaction was “it comes with the job”). Now I work tech full time, and I’m a deckhand when I’m not doing that.
My perspective is shaped by a couple things, I think:
Being born to teenage parents who worked similar jobs and struggled for a long time
The fact that they “raised me right” – if I talked to / treated anyone the way I’ve seen some folks I’ve met in this industry do to service workers / people they seem to consider as “below them”, they wouldn’t be having any of it
Actually working the jobs and being subject to the awful treatment by both customers and management
The thing is, though, is that I really don’t think you should have to go through any of this in order to just not be a jerk to people…I really don’t know what the disconnect is. The most difficult customers I’ve had (at previous jobs and on the boat) have typically been the ones that seem the most privileged. When it comes to restaurants, the cheapest ones (in terms of tipping) were similarly the people that would come in and order hundreds of dollars worth of food and then leave little to no tip (I’m not here to debate tipping culture, it is what it is here at this point).
I’ve had situations where I take people to a place where I’m cool with the staff and someone picks up the tab (for which I’m appreciative) but then they are rude / pushy / skimp out on the tip, which is really embarrassing to say the least (I don’t hesitate to call people out but I feel like … I shouldn’t have to?)
The boat I work on is in the bay area and so we get a lot of tech people, and a couple of things stand out:
I don’t really know how some of the most intelligent people can be so dumb (literally all you need to do is follow directions)
They talk down to us (the crew trying to put them on fish and, for what it’s worth, keep them alive – won’t get into that here), and when you ask them not to do something for safety or you try to correct something they’re doing wrong, they get an attitude. I want to emphasize, not everyone, but enough to make you stop and ask why.
When they find out that I also work in tech (you talk to people since you’re with them for 8+ hours), the reaction is typically one of “why do you need to be doing THIS?”. Sidenote – the most hilarious thing that I enjoy doing is dropping into a technical conversation (a lot of people come on with their coworkers) and having people be like “wtf how does the deckhand know about any of this?”
They don’t tip … lol … or they stiff us for fish cleaning which we are upfront is a secondary service provided for a fee.
It’s not everyone, but I get a pretty decent sample size given the population here. The plus side of working on the boat (vs a restaurant or retail) is that if someone starts being a major a-hole the captain doesn’t mind if we stand up for ourselves (encourages it, even)
It’s not everyone of course, but it’s enough to make you wonder.
Yeah, exactly. Or, we have a saying “you can tell whose never pushed a broom in their life”.
That was more of a rant than I wanted to get into but since I’m here it was kind of cathartic. I really just wish people would stop and think about the human on the other end. Of course it’s not just tech people that do things like this, but … yeah.
I worked in eDiscovery for a while (~3 years) so I have a sense of legal. It’s very stratified and stressful. I remember going to bed at 2 AM and waking up at 5 AM to make sure that a production was ready for opposing counsel. Not ideal…
By contrast, my father was 39 when he had me. However, he had a hard life. He grew up in Francoist Spain. (One of the few memories of my grandfather was when he told me “La habre es miseria. La habre es miseria.” (Hunger is misery. Hunger is misery.)) My father was a Spanish refuge in France at age 9. He didn’t complete high school. Instead, he did an apprenticeship in a French restaurant where the chefs beat him. He worked 16 hour days for a long time.
Absolutely. My father always said that everyone was welcome at his restaurant, regardless of what they were wearing. It’s important to respect everyone.
Inter-generational trauma is a real thing. I’m doing okay, but my brothers didn’t fare so well. (A topic for a more one on one conversation.) I hope you are okay. <3
Edit: this has really thrown me through a loop. I don’t mean to be dramatic and I know this is a public forum, but I’m sure there are more people posting than responding. If it means anything to anyone then it’s more important to say so than to be stoic. I hope you are all doing okay.
Heh, sorry I meant legal vs not-so-legal in the legality sense, but in any case wow that sounds dreadful!
I appreciate your kind words and you sharing your story, and I’m glad you’re doing okay. I’m also sorry to hear about your brothers, similar thing is true for some of my siblings…kind of weird how that works.
Thank you for sharing this.
I meant to write “more people reading than responding” above, but I’m out of the edit window.
There is a theory in some circles that states that having money enable people to not have to care about other human beings. With money you can feel like you provide for all your needs by just buying stuff and services. If you don’t have so much money, you need to compensate by trying to build mutual understanding. That leads to being more empathic. You also need to respond or even anticipate the needs of people who give you money. Which leads also to some kind of asymmetric empathy (similar to impostor syndrome). Also there may be the fact that some people are attracted to tech because they fell they are more gifted with machines than with people. So maybe some form of selection bias here too.
I like to remind my team something that I was once told: “Remember, this work lets us have soft hands.”
Always reminds me of that scene in Trading Places (the “soft hands” part is cut off at the beginning).
Well put. I sometimes ask myself, “How many people are living miserable lives so that I can sit in a cushy chair and think about interesting problems?”
How many people’s misery could you alleviate by switching to a different job and how would that happen?
Well, I’ve worked in the oil and gas industry, so I helped keep lots of people’s heat working in the winter, including my own. At the cost of making the world incrementally more fucked though, so that one’s a net negative. I’ve done a fair amount of teaching, so I helped share skills that were useful for people. I’ve worked datacenter tech support, so I helped lots of people keep their online businesses working. So there’s that.
If I really wanted to make the world a better place, I would either work for something like Habitat For Humanity and build houses, or I would get a PhD in nuclear physics or material science and try to make better nuclear/solar energy sources. Or become a teacher, natch, but my sister and both parents are teachers so I feel like I have to break the family mold. Could always do what my grandmother did and become a social worker. Or go into politics, because someone with a brain stem has to, but I’ve had enough brushes with administration to know that I will burn myself out and be of no use to anyone.
Right now I work in robotics doing R&D for autonomous drones, so I’ll hopefully make peoples’ lives better somewhere, someday down the line. Nothing as good as what Zipline does, but on a similar axis – one of my most fun projects was basically Zipline except for the US Army, so it was 10x more expensive and 0.25x as good.
…do people not normally think about this sort of thing?
Interesting, that’s not the way I interpreted cole-k’s comment!
…how did you interpret it?
That there are people supporting cole-k’s job (I don’t know who, maybe car mechanics, cafeteria workers, janitors?) whose work is required for cole-k’s job to be possible, but who are necessarily miserable in their jobs.
Yeah, and I’ve done at least a moderate share of supporting them back one way or another, within the bounds of my aptitudes, skills, and how selfish I’m willing to be – and honestly I’m pretty selfish, ngl. Sometimes I’ve done it directly by serving them back, more often indirectly by trying to do things that Benefit Everyone A Bit. All I can do is keep trying. We’re all in this together.
This should not be controversial, and sometimes I wish I had a button to teleport some of my colleagues where I used to work in Africa to recalibrate their sense of what “hard” means.
This is so true. I try to remind myself of this as much as I can but as I did not experience minimum wage work myself this can be hard to be fully aware of this situation. Maybe we should try for the condition of everybody to improve. I fear that by insisting a lot on the good conditions we have in the tech industry it would only encourage a degradation on those conditions unfortunately. Tactically, I wonder if we should not focus on the commonalities of the issues we face across all the different types of jobs.
I also paid for college and university working in a large hotel kitchen and then dining room. At the time in the front of the house I could earn enough in tips over a summer to cover a year of state school tuition plus room and board. I’d go back on holidays and long weekends to make the rest of my expenses. It was hard work, long hours, and disrespected in all kinds of ways. Once in a while there was violence or the threat of violence. But it beat doing landscaping or oil changes and tires. There were a number of us who were using it as a stepping stone, one guy from Colombia worked until he saved up enough to go back home and buy a multi-unit building so he and his parents could live in it and be landlords, get a used BMW for himself, and finish his education. His motivation, and taking extra shifts, made mine look weak and I was highly motivated.
I remind myself of that time when I’m frustrated at my desk.
Colombia, or do you mean he was studying at Columbia?
Typo. Fixing.
This I think was one of Bryan Caplan’s arguments about open borders.
In addition to the moral issue that no-one has the right to curtail the free movement of others[1], there is solid empirical evidence that not only do immigrants enrich the countries they emigrate to (i.e. contribute more on average than locals), they often also help lift their home countries out of poverty by doing exactly what your Colombian friend did.
Edited to add: here’s his address on the topic of poverty: https://www.youtube.com/watch?v=K77cGFU36rM
[1] One frequently occurring example of hypocrisy on the matter of travel: people who simultaneously rail against any attempt by their own Government to control their movement (passports, papers, ID, etc.), but also complain loudly about people crossing the border into “their” country and demand the Government build a wall, metaphorically or literally.
I think I prefer the way Elixir does things: https://fly.io/phoenix-files/single-file-elixir-scripts/
You can do it as well:
Does that use a temporary venv?
Nope. It will use whatever interpreter you launched the script with
Thx. That makes it inappropriate for this use case, imo
Requiring a full Python parser for metadata is not great.
Plus this will fail in many situations because you might now be in a venv.
Finally, if the packages are installed, this will make the script slow.
You would need a lot of if for this.
The same can be said about the Elixir example
Indeed, and Elixir being even more exotic, this limits the metadata parsing a lot.
I’ve encountered this use case, it would be handy, but adding magic to comments is always dangerous. I would rather include a requirements.txt and give a way to have “ensure these requirements and then run” in a shebang
In the meantime… you can run pip directly at the start of the script, run pip install in a directory, node_modules style, change sys.path and continue with the script.
If you have 10 scripts in a directory, with different requirements, you get to have 10 different requirement files. I think there is a place for this.
Metadata in comments? I don’t really adhere to this.
I’d rather ship a requirements.txt alongside the script. There is no need to complexify things.
I try to defend Python packaging when people say it’s a mess, but PEP like this one don’t help me :/
Yeah, the use case of “I want to write a script and distribute it, but not involve any proper packaging” should really just consist of handing someone a zip file with the script and some instructions on how to set it up, rather than trying to get the packaging ecosystem to support it.
This is what PEX solves.
Chose zipapp over pex, it’s standard and more portable (see my other comment)
PEX by default builds a (better) zipapp. The resulting executable is more portable than a regular zipapp because it bundles an interpreter with versioned dependencies and executes it in an isolated environment. I would literally never choose to use a zipapp over a pex file.
This has already been answered other comments: it’s solved problem, but this solution only addresses partially the issue this PEP targets.
See: https://lobste.rs/s/i0au4g/pep_722_python_dependencies_for_single#c_9gqdqy
I read that comment. And I don’t think the “but what about C extensions” bit is relevant. Somebody who has a single-file script with a C extension that they want to distribute to others without properly packaging it… is someone I’m happy to leave unsupported. Packaging is easy enough at this point that for any code that’s even moderately complex (and a C extension is way past that point) it should just be the default. And for the single-file script whose author refuses to use proper packaging tools, zipapp ought to be the way to go. Which leaves no space for this proposal.
In that case you are leaving in the dirt
beginners, students, kids, that just do small things and installed pillow or pygame and wanted to share that with their friends but barely know the ecosystem
sysadmin that want a quick and dirty script a thousands time a day to check something out on machine x that happens need a compiled driver to access z and they work on windows but test on linux
thousands of professionals that are not coders, like biologist, mathematicians, geographers and analysts that just know a little bit of python and have neither the time, desire or skill to learn an entire ecosystem to make share their matplot lib graphs with their colleagues by email
This reads like somebody living in the bubble of pro dev, forgetting a gigantic part of the Python community is not. That’s the appeal of the language. And we should cater to them to. They suffer enough with packaging.
And as a professional Python dev that started in 2.4, I would be very happy with having a way to add things like fabric with on line in my numerous scripts that I touch once a year. I don’t want to have requirements files everywhere for 50 modules of 20 lines that have been created over 15 years and have no congruency whatsoever.
I would pay real money to never again have someone bust out the “if you don’t support this exact proposal, then you must not care about beginners and students!” fallacy ever again.
The zipapp technique already exists, and it works for this use case (“I have a script, I don’t want to learn packaging, just let me send it to someone else”). You yourself admit that it works for this use case. The case where it doesn’t work is the case of also wanting to include a custom compiled C extension with the script, and if you think beginning Python programmers are routinely doing that, then I’d be interested to know exactly who your “beginners” are.
And let’s face it: if the complaint is that, say, learning to make a requirements file in order to reproduce an environment is too difficult and complex and too much effort, then how on earth is “let’s make them learn to make a requirements file, we’ll just store it in a different place” an acceptable alternative?
It seems that for some reason you like this proposal. But you need to A) be honest about the alternatives, B) be honest about its flaws and C) understand that other people may also be aware of the issues faced by beginners, students, academics, etc. but may reasonably come to different conclusions about how best to handle those use cases.
There is already a non comment-base solution to this problem: zipapp (https://docs.python.org/3/library/zipapp.html).
Nobody uses them. Nobody even know they exist.
Thanks to c-extensions, it’s not cross platform. And they require your source to be zipped, which is less than ideal if you edit it often.
This proposal has several benefits:
it’s simple enough beginners can understand and use it
it’s clear enough people will be able to share around it. zipapp are not clear
it works cross platform
it doubles as documentation
it solves the “I have many small scripts with different requirements in the same dir” problem, which is common for the part of the community doing data analysis, finance, biology and sysadmin. A.K.A pros coding in python, but not pro python devs
Personally, I like it a lot.
When I’m in a project, I don’t care.
But at home, I have a Scripts dirs with a lot of small quick and dirty 20 utility utilities. I have to avoid using requests, keyring or typer because dealing with deps on those would be annoying. This would solve that.
Well, I mostly switched from Python to Ruby (with the inline Bundler mode, which this PEP tries to replicate – badly, in my opinion) for scripts not because I haven’t heard of zipapp (actually, I haven’t, thank you; I heard of PEX, though), but because I don’t want the extra hassle of dealing with dependency management outside the script. Zipapp solves the problem by requiring me to package my scripts into archives. You highlight this and other problems yourself.
If I need to run an extra step to run each of my scripts, why not use a compiled language like Nim then? That way, once my script compiles, it will always work without a fault from
/usr/local/bin. Which cannot be said about Python/Ruby scripts when system packages upgrade.At the same time, the inline Bundler mode in Ruby actually reduces the burden of dependency management by ensuring that when I run
ruby script.rb, I have one less thing to worry about. And, as opposed to this PEP, inline Bundler config is (a) real Ruby code, and (b) uses the same syntax in the inline and Gemfile (standalone) modes.But it will be difficult to make one-liner script with more than one file