1. 65

I haven’t shared my own content here before but I feel like this is worth it because it’s also a sincere offer to mentor and grow new speakers that would make Python conferences more interesting again.

  1.  

  2. 7

    I’m currently a Python dev (apparently this is the most recent turn my career has taken), and I’m really bummed out by its web story outside of Django.

    My last gig was Elixir, before that Node, and some Rails and Laravel in there. The tooling in the Python ecosystem, especially around migrations and dependency management, just feels clunky.

    It singlehandedly sold me on Docker just so I didn’t have to mess with virtualenvs and multiple runtimes on my system and all of that. Like, what happened? Everybody groused about 2-to-3 (which is still hilarious) but like even without that I feel like the ecosystem has been vastly outstripped by “worse” technologies (see also, NodeJS).

    1. 4

      It singlehandedly sold me on Docker just so I didn’t have to mess with virtualenvs

      One thing that made virtualenvs almost entirely painless for me was using direnv: in all my python project directories I have a bash script named .envrc that contains source .venv/bin/activate, and now cd-ing in/out of that directory will enter/exit the virtualenv automatically and instantaneously. It’s probably possible to set it up to switch pyenv environments as well.

      1. 3

        One of the reasons why Python packaging still feels so clunky compared to other ecosystems is that the Python ecosystem is a lot more diverse thanks to e.g. the scientific stack that has very different needs than the web peeps so there’s never gonna be an all-encompassing solution like Cargo. Pipenv tried and failed, poetry is carving a niche for itself.

        But the primitives are improving. pip is currently growing a proper resolver and doesn’t e.g. Ruby still need a compiler to install binary packages? As long as long as you don’t use Alpine for your Docker images, Python’s wheels are great (they’re just a bit painful to build).

        1. 1

          How did pipenv fail?

          1. 4

            Short answer: it’s too complex which makes it buggy and there wasn’t a release in over a year. IOW: It’s falling over it’s own weight.

            Long answer: https://hynek.me/articles/python-app-deps-2018/

        2. 3

          The tooling in the Python ecosystem, especially around migrations and dependency management, just feels clunky.

          Currently working on a Rails app, coming from the Flask ecosystem. You have no idea how much I can miss SQLAlchemy and Alembic.

          I agree about dependency management, but certainly not about migrations. Modifying models and auto-generating migrations works much better than the other way around for me.

        3. 3

          LOVE this article! Especially the offer to help folks in need to get more visibility. Good on you for being willing to put elbow grease into moving the needle!

          A question about your reactions to people writing about building in the cloud. Are you saying you’d like to see less of that, or that you think doing so is a bad idea to begin with?

          I ask because the former seems a perfectly reasonable preference, but I’d argue that the latter could be a reactionary stance we might think carefully before taking.

          The cloud is a GREAT tool for certain use cases and an AWFUL one for others. I’d love to see some of the hype and acrimony get stripped away so we could all just use the right tool for the right job and get on with our lives :)

          Your chances of me helping are increased if you’re part of an URM and/or if I find your topic interesting. Please accept my apology if I can’t help you specifically, but I’ll try to find time for as many people as my time permits.

          Just in case anyone else read this and feels too abashed to ask: URM is an acronym for Under Represeented Minority.

          1. 1

            A question about your reactions to people writing about building in the cloud. Are you saying you’d like to see less of that, or that you think doing so is a bad idea to begin with?

            Not at all! I’m just somewhat annoyed by the fact that given a lot of public discourse is dominated by paid cloud advocates you could get the impression, that everyone is is running their stuff in their clouds, on top of their products like hosted Kubernetes.

            That’s obviously wrong but they’re paid to give you that impression and I suspect that many people feel inadequate due to that despite having good reasons to run their stuff differently. We need to get those people too – not asking for exclusivity. :)

            1. 1

              I think you’re right, and I also think some people end up feeling like they SHOULD run their workloads in the cloud even when maybe doing an actual evaluation of their situation might serve them better. Heck, a cloud solution may well be exactly the right fit, but you won’t know until you really survey all the options and figure out what works best for you.

          2. 2

            I want to write an article on this topic for a long time, because I’ve noticed same problem as you. I’ve even seen people talking about how python is not good language for running the production.

            1. 2

              We are one of those Python web services companies running on Docker (AWS Fargate). One of our big problems has been dependency management in our monorepo. We want to be able to build and deploy to production quickly and often (more than a dozen times per day) so we want our CI job to run in ~10 minutes or less and our deploys to run in ~30 minutes or less. We also care about reproducible builds, so we initially looked at pipenv, but it took 30 minutes just to resolve dependencies for any change in any container. Eventually we moved to https://github.com/pantsbuild/pants which has solved many problems, but it’s an awful piece of engineering that happens to do the job right so long as you never deviate from the happy path and you don’t need to do reasonable things like ask, “what is the unique hash of this version of my system? [so I can use that to tag my Docker images]”. In general, dependency management and ecosystem tooling is still a big pain and we spent a lot of time to find something that worked only passably. Others have probably found a solution that works well, but we haven’t stumbled upon it yet (possibly because there isn’t enough attention devoted to running web services among the Python community?).

              1. 9

                Without knowing your details, I can’t give advice, but as a data point: I’ve personally found joy in the flexibility of pip-tools. I’ve blogged about it (well, I mostly blogged about why I don’t use Pipenv and poetry) in 2018 and updated it November 2019 with my personal workflow: https://hynek.me/articles/python-app-deps-2018/

                1. 2

                  I haven’t even heard of pip-tools. Reading through your blog post now. Thanks for the recommendation!

              2. 1

                I’ve personally found Poetry + Docker to be a really effective combination for building and deploying “web apps”. My audience is small, myself & a few other engineers, so I haven’t had to worry at all about scale, but I’ve found it incredibly easy to maintain my application. Need to update a dependency (poetry show -o)? Just change pyproject.toml and run poetry update. If you want to be fancy with your docker caching do poetry export -f requirements.txt. Don’t forget your docker build afterwards (or commit and let CI build for you).

                As a bonus, Poetry makes it easy to install into a virtual environment on my development machine, which VSCode (my current editor of choice) can easily pick up and use for type completion and other utilities (formatters, linting, etc), so long as those dependencies are specified as dev-dependencies in pyproject.toml.

                Previously I had been hand maintaining virtual environments and requirements.txt files and pulling my hair out trying to manage dependencies. Poetry has been an absolute game changer.