1. 29

  2. 5

    I agree with optimizing Python’s startup time, and I agree that namedtuple is weird and should be changed.

    But I doubt that the namedtuple change will noticeably decrease startup time for most applications (after having experience with this problem for 10+ years).

    From a comment 3 years ago:

    [Python interpreter startup does] random access I/O (the slowest thing your computer can do) proportional to (large constant factor) x (num imports in program) x (length of PYTHONPATH).


    I don’t think that instantiating a Python parser 100 times for exec() is going to compare to that. I guess the difference is that I’m thinking about the cold cache case – namedtuple might be noticeable in the warm cache case.

    And there are many many command line tools that start slow because they’re written in Python, not just Mercurial.

    Mercurial is actually one of the best because they care about it (demandimport) and they don’t have too many dependencies. IIRC some of the Google Cloud tools take 500-1000+ ms to start because they are crawling tons of dependencies with long PYTHONPATH.

    (copy of Hacker News comment)

    1. 5

      Mercurial is actually one of the best because they care about it (demandimport) and they don’t have too many dependencies.

      And even under those circumstances, the maintainers considered chg [1] a sufficiently important improvement that they included it in core when Facebook built it.

      [1] chg reduces response time by not starting up a new Python interpreter for every command. It starts one Mercurial process that stays alive, and makes the command-line hg a C shim that starts quickly and talks to the backgroundhg process.

    2. 2

      Let the data be your guide. http://i.imgur.com/jKu7x6q.png

      Apparently python chews on the filesystem pretty hard and mallocs a bunch. I would imagine the pyinstaller style things that produce executables would reduce the first problem because the byte code for a given module is just shoved into the exe/elf.

      1. 2

        I’m just happy this might make namedtuple’s implementation sane. That weird use of eval prevents them from being pickle-able, which every now and then causes issues when I use some library that has namedtuples somewhere deep inside.