thinktoomuch.net

An Emerging Memetic Engineer - Looking for the Good in Everything

thinktoomuch.net header image 2

Python Call Graphs

June 6th, 2007 · Posted by Who Knows? · 5 Comments

Traditional blogging wisdom would say I should have more than one blog, to keep the subject matter more “focused”. Well, oops. What can I say. I’ll sort that out when my thesis is done. (Like the rest of my life. ;) ) In the mean time, I’ll always feel some urge to not post more technical stuff, like this. (Also, I suppose it doesn’t make sense to have this kind of post imported into Facebook - if you’re reading this in Facebook, would you prefer not to see “notes” like these?)

I found a cute call graph generator for python, pycallgraph. Despite still needing some attention (which it doesn’t seem to be getting?), it is usable, once you fix a major defect by adding a single line:

Index: pycallgraph.py
===================================================================
--- pycallgraph.py      (revision 23)
+++ pycallgraph.py      (working copy)
@@ -182,6 +182,7 @@
     if event == 'return':
         if call_stack:
             call_stack.pop(-1)
+    return tracer

 def get_dot(stop=True):
     """Returns a string containing a DOT file. Setting stop to True will cause

I’m not sure why this problem exists… this bug (and fix) has been known for quite some time? The actual graph is drawn with graphviz. There are some examples on the website, but of course, it’s most fun to run it on your own code that you know well. Usage (once fixed and installed) is as simple as adding four lines of code:

import pycallgraph
pycallgraph.start_trace()
...your code here...
pycallgraph.stop_trace()
pycallgraph.make_dot_graph('test.png')

This will use “dot”. You can use other graphviz binaries and adjust the output format:

pycallgraph.make_dot_graph('test.jpg', format='jpg', tool='neato')

I’m not sure what the best way would be to collect a trace over all your unit tests. I placed start_trace(reset=False) and stop_trace() in each test (placing them in setUp() and tearDown() would require filtering out the testing framework if I want a clean graph, and I’m a little lazy to use pycallgraph’s exclusion mechanism right now). Then I subclassed unittest.TextTestRunner to include make_dot_graph(...) as follows:

class CallGraphTextTestRunner(TextTestRunner):
    def __init__(self, callgraph_filename, *args, **kwargs):
        TextTestRunner.__init__(self, *args, **kwargs)
        self._callgraph_filename = callgraph_filename

    def run(self, test):
        result = TextTestRunner.run(self, test)
        pycallgraph.make_dot_graph(self._callgraph_filename)
        return result

main(testRunner=CallGraphTextTestRunner('test_callgraph.png'))

What an unimaginative class name. Any better suggestions?

(If you are new to unit testing in Python, you might need to look at more than just a basic example of unit testing in Python.)

Categories: Technology · Website
Tags:

5 responses so far ↓

  • 1 Hugo // Jun 6, 2007 at 6:54 pm

    Of course, generating call graphs hooks into all function calls (using sys.settrace), so this is rather slow. You would want to do it rather selectively, for specific debugging or overview purposes, thus the above is of course overkill.

    I’m still looking for the nicest code coverage solution…

    OK, enough playing, back to work!

  • 2 Hugo // Jun 6, 2007 at 7:24 pm

    Naah, one last comment… timing:

    Two rather simple tests that effectively make 28 function calls…

    Standard test running time: 0.001s
    Test running time with one start_trace call, no tracing: 0.085s
    Test running time with two start_trace call, no tracing: 0.085s
    Test running time with two start_trace call, tracing 4 function calls: 0.25s
    Test running time with two start_trace call, tracing 28 function calls: 1.25s

    Scary…

    And then I’ve seen another bug… it seems calls to start_trace(reset=False) loses edges from __main__ for all but the first start_trace, i.e. the first one after a reset_trace(). It seems maybe there is something that needs “resetting” even when not resetting. ;)

  • 3 Hugo // Jun 7, 2007 at 1:41 pm

    Sweet! The developer *is* responsive! The above one-liner fix has been included in subversion. Submitting another feature request… ;)

  • 4 Gerald Kaszuba // Jun 9, 2007 at 3:15 pm

    Hi Hugo, I’ve released 0.3.1 to fix that bug ;)

    I’ll get to work on your off-line filtering ticket soonish.

  • 5 Prashanth Ellina » Blog Archive » Generating call graphs for understanding and refactoring python code // Nov 14, 2007 at 5:06 am

    [...] http://thinktoomuch.net/2007/06/06/python-call-graphs/ [...]