Summary

Profiling Python is not always an easy task, but I hope I've covered some of the tools you might use. Before using any of the tools, be sure you know how it does the profiling – deterministic or statistical – and what it is profiling – the entire code or just a function.

Although I didn't talk much about putting timing points in code (manual profiling), I'm a bit old school. That level of control lets me gather timing data for various portions of code pretty easily. If you are old school like me, you are probably already using this method in your code. If you haven't done it before, I suggest giving it a try.

In using one or more of the profiling tools, I suggest putting code in functions and profiling those functions deterministically, if possible, so you can isolate various parts of a program. While you are isolating parts of your code in functions, why not take advantage of the situation and look at using Numba to compile these functions? The speed-up obtained can be pretty amazing.