1. 8
  1.  

  2. 4

    This is the nicest profiler I’ve found for Python. It’s cross-platform, doesn’t need me to change the code that’s being profiled, and its output can be piped directly into flamegraph.pl. I just used it to pinpoint a gross misuse of SQLAlchemy at work that’s run in some code at the end of each day, and now I can go home earlier.

    1. 1

      No mention in the readme of how it handles (or doesn’t) forks or threads. Would be curious to hear more!

      1. 2

        It handles threads just fine, and doesn’t handle multiprocessing done via spawning (which isn’t surprising; it’s a sampling frame stack profiler so it only has access to what the parent process does). It’s not happy under WSL so I can’t check what it does for children that fork, but maybe some kind soul can run the following on Linux and report?

        import time
        import multiprocessing
        
        def fact(n):
            f = 1
            for i in range(1, n+1):
                f *= i
            return f
        
        def do(N):
            n = 1
            for _ in range(N):
                fact(n)
                n += 1
        
        if __name__ == '__main__':
            threads = []
            for _ in range(2):
                t = multiprocessing.Process(target=do, args=(5000,))
                t.start()
                threads.append(t)
        
            for t in threads:
                t.join(timeout=5)
        
        1. 2

          (Did I say an hour? Time flies…) Running that code, it looks like the traces just find a load of processes in their popen join/wait loops. No trace of fact or do

          1. 2

            That’s the same as for the spawning children. There’s our answer: threads ok, not-threads not ok.

            1. 1

              Makes sense, thanks!