The State of Python Profiling
2018-10-12
There are no good python profiling tools. Here is what I consider necessary from a profiler:
- Cross platform
- Profiling of live applications
- Ability to generate profiler dumps (preferably gprof compatible)
- Call tree profiling
- Line-level profiling
The built-in python profilers cProfile and profile don’t meet 1, 2, and 5. 3rd party alternatives also fall short.
Pyflame
A great live profiler:
- low overhead.
- live profiling of processes.
- generates profiler dumps, convertible to gprof and svg.
- Can generate Chrome profiler dumps.
but it has some limitations
- Uses linux’s strace so it’s not cross platform.
To complicate matters:
- strace is not enabled by default on docker containers.
- strace is disabled for regular users on a lot of distros.
Vprof
- Linux only
line_profiler
- Uses cProfile so it shares its limitations
References
https://kupczynski.info/2015/01/16/profiling-python-scripts.html