Profiling Python code

In Profile

To improve the performance of your applications, you need to conduct some kind of dynamic (program, software, code) analysis, also called profiling, to measure metrics of interest. A key metric for developers is time (i.e., where is the code spending most of its time?), because it allows you to focus on areas, or hotspots, that can be made to run faster.

And, this might seem obvious, but if you don't profile for code optimization, you could flounder all over the code improving sections you think might be bottlenecks. I have seen people spend hours working a particular part of their code when a simple profile showed that portion of the code contributed very little to the overall run time. I admit that I have also done this; however, once I profiled the code, I found that I had wasted my time and needed to focus elsewhere.

Different kinds of profiling (e.g., event-based, statistical, instrumented, simulation), are used in different situations. In this article, I focus on two types: deterministic and statistical. Deterministic profiling captures every computation of the code and produces very accurate profiles, but it can greatly slow down code performance. Although you achieve very good accuracy with the profile, run times are greatly increased, and you have to wonder whether the profiling didn't adversely affect how the code ran. For example, did the profiling cause the computation bottlenecks to move to a different place in the code?

Statistical profiling, on the other hand, takes periodic "samples" of the code computations and uses them as representations of the profile of the code. This method usually has very little effect on code performance, so you can get a profile that is very close to the real execution of the code. You do have to wonder about the correct time interval to get an accurate profile of the application while not affecting the run time. Usually this means setting the time intervals to smaller and

... comments powered by Disqus