1. 4
    Sampling profiler for Python programs programming python github.com
    awn avatar via awn 3 years ago |
    Archive.org Archive.today Ghostarchive
    | 4 comments
    1. 2
      inactive-user avatar inactive-user 3 years ago | link

      Ooh ooh!!! I can totally plug this in!

      Now if we only had one for memory usage that recursively broke it down by types from top level containers! That would make tracking production memory hogs easier!

      1. 3
        emeryberger avatar emeryberger 3 years ago | link

        Not sure if breaking down by types is necessarily that helpful, but Scalene not breaks down CPU and memory usage on a per-line basis.

        Memory usage: ▂▂▁▁▁▁▁▁▁▁▁▅█▅ (max: 1617.98MB)
phylliade/test2-2.py: % of CPU time =  40.68% out of   4.60s.
       |    CPU % |    CPU % |  Net  | Memory usage   | Copy  |
  Line | (Python) | (native) |  (MB) | over time /  % | (MB/s)| [phylliade/test2-2.py]
--------------------------------------------------------------------------------
     1 |          |          |       |                |       | import numpy as np
     2 |          |          |       |                |       | 
     3 |          |          |       |                |       | @profile
     4 |          |          |       |                |       | def main():
     5 |          |          |    92 | ▁▁▁▁▁▁▁▁▁  11% |       |     x = np.array(range(10**7))
     6 |    0.43% |   40.24% |   762 | ▁▁▄█▄      89% |   168 |     y = np.array(np.random.uniform(0, 100, size=(10**8)))
     7 |          |          |       |                |       | 
     8 |          |          |       |                |       | main()
        1. 3
          inactive-user avatar inactive-user 3 years ago | link

          Oh my god you don’t know how much this means to me. I’ve been trying to track down memory issues for months due to heavy objects from the database and a possible leak - this is exactly what I was looking for to put into use! I can now have a periodic dump suitable for stdout scraping. Thank you!

          1. 2
            emeryberger avatar emeryberger 3 years ago | link

            Great! Let me know how it goes (I am working on making the malloc-tracking functionality even faster, too).

            One thing: relying on the ASCII art output is going to be pretty brittle. It wouldn’t be hard to have this info be generated in CSV or JSON format (I’d prefer the latter). If that sounds helpful, please don’t hesitate to open an issue!

Stories with similar links:

  1. Py-Spy: A sampling profiler for Python programs via Arnie0426 5 years ago | 19 points | 2 comments