How can I profile Python code line-by-line?

I believe that's what Robert Kern's line_profiler is intended for. From the link:

File: pystone.pyFunction: Proc2 at line 149Total time: 0.606656 sLine #      Hits         Time  Per Hit   % Time  Line Contents==============================================================   149                                           @profile   150                                           def Proc2(IntParIO):   151     50000        82003      1.6     13.5      IntLoc = IntParIO + 10   152     50000        63162      1.3     10.4      while 1:   153     50000        69065      1.4     11.4          if Char1Glob == 'A':   154     50000        66354      1.3     10.9              IntLoc = IntLoc - 1   155     50000        67263      1.3     11.1              IntParIO = IntLoc - IntGlob   156     50000        65494      1.3     10.8              EnumLoc = Ident1   157     50000        68001      1.4     11.2          if EnumLoc == Ident1:   158     50000        63739      1.3     10.5              break   159     50000        61575      1.2     10.1      return IntParIO

You could also use pprofile(pypi).If you want to profile the entire execution, it does not require source code modification. You can also profile a subset of a larger program in two ways:

• toggle profiling when reaching a specific point in the code, such as:

  import pprofileprofiler = pprofile.Profile()with profiler:    some_code# Process profile content: generate a cachegrind file and send it to user.# You can also write the result to the console:profiler.print_stats()# Or to a file:profiler.dump_stats("/tmp/profiler_stats.txt")

• toggle profiling asynchronously from call stack (requires a way to trigger this code in considered application, for example a signal handler or an available worker thread) by using statistical profiling:

  import pprofileprofiler = pprofile.StatisticalProfile()statistical_profiler_thread = pprofile.StatisticalThread(    profiler=profiler,)with statistical_profiler_thread:    sleep(n)# Likewise, process profile content

Code annotation output format is much like line profiler:

$ pprofile --threads 0 demo/threads.pyCommand line: ['demo/threads.py']Total duration: 1.00573sFile: demo/threads.pyFile duration: 1.00168s (99.60%)Line #|      Hits|         Time| Time per hit|      %|Source code------+----------+-------------+-------------+-------+-----------     1|         2|  3.21865e-05|  1.60933e-05|  0.00%|import threading     2|         1|  5.96046e-06|  5.96046e-06|  0.00%|import time     3|         0|            0|            0|  0.00%|     4|         2|   1.5974e-05|  7.98702e-06|  0.00%|def func():     5|         1|      1.00111|      1.00111| 99.54%|  time.sleep(1)     6|         0|            0|            0|  0.00%|     7|         2|  2.00272e-05|  1.00136e-05|  0.00%|def func2():     8|         1|  1.69277e-05|  1.69277e-05|  0.00%|  pass     9|         0|            0|            0|  0.00%|    10|         1|  1.81198e-05|  1.81198e-05|  0.00%|t1 = threading.Thread(target=func)(call)|         1|  0.000610828|  0.000610828|  0.06%|# /usr/lib/python2.7/threading.py:436 __init__    11|         1|  1.52588e-05|  1.52588e-05|  0.00%|t2 = threading.Thread(target=func)(call)|         1|  0.000438929|  0.000438929|  0.04%|# /usr/lib/python2.7/threading.py:436 __init__    12|         1|  4.79221e-05|  4.79221e-05|  0.00%|t1.start()(call)|         1|  0.000843048|  0.000843048|  0.08%|# /usr/lib/python2.7/threading.py:485 start    13|         1|  6.48499e-05|  6.48499e-05|  0.01%|t2.start()(call)|         1|   0.00115609|   0.00115609|  0.11%|# /usr/lib/python2.7/threading.py:485 start    14|         1|  0.000205994|  0.000205994|  0.02%|(func(), func2())(call)|         1|      1.00112|      1.00112| 99.54%|# demo/threads.py:4 func(call)|         1|  3.09944e-05|  3.09944e-05|  0.00%|# demo/threads.py:7 func2    15|         1|  7.62939e-05|  7.62939e-05|  0.01%|t1.join()(call)|         1|  0.000423908|  0.000423908|  0.04%|# /usr/lib/python2.7/threading.py:653 join    16|         1|  5.26905e-05|  5.26905e-05|  0.01%|t2.join()(call)|         1|  0.000320196|  0.000320196|  0.03%|# /usr/lib/python2.7/threading.py:653 join

Note that because pprofile does not rely on code modification it can profile top-level module statements, allowing to profile program startup time (how long it takes to import modules, initialise globals, ...).

It can generate cachegrind-formatted output, so you can use kcachegrind to browse large results easily.

Disclosure: I am pprofile author.

You can take help of line_profiler package for this

1. 1st install the package:

    pip install line_profiler

2. Use magic command to load the package to your python/notebook environment

    %load_ext line_profiler

3. If you want to profile the codes for a function then
do as follows:

    %lprun -f demo_func demo_func(arg1, arg2)

you will get a nice formatted output with all the details if you follow these steps :)

Line #      Hits      Time    Per Hit   % Time  Line Contents 1                                           def demo_func(a,b): 2         1        248.0    248.0     64.8      print(a+b) 3         1         40.0     40.0     10.4      print(a) 4         1         94.0     94.0     24.5      print(a*b) 5         1          1.0      1.0      0.3      return a/b