Wednesday, April 28, 2010

Debugging Python code from within Emacs on Windows

It is very easy to debug Python code from within Emacs. However, it does not work as advertised on my Windows XP development machine - I do not know whether it works as advertised on Linux. The problem is that the Emacs buffer to interact with the Python debugger (Pdb) only displays the directory of the source file I want to debug:

Current directory is c:/projects/customer X/scripts/converter/converter

In case the problem is related to the Emacs version, I am using version 23.1.1.

After some investigations, I found out that there two things I had to take care of to get the interaction with Pdb working.

1. To start Pdb from within Emacs, you M-x the command "pdb" (without the quotes). You then have to enter the command-line required to start Pdb. On my Windows XP machine that is

python c:/Python26/Lib/pdb.py updateweights_tests.py

Emacs parses the output of Pdb to stdout to let the user interact with Pdb. By default, this output is buffered on Windows and Emacs can only parse the part of the output that has been flushed. This can result in a deadlock where Emacs is waiting for more PDb output and Pdb for user input.

To turn off buffering, you have to supply the Python interpreter with the command-line option -u:

-u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
see man page for details on internal buffering relating to '-u'

The command-line to start Pdb becomes

python -u c:/Python26/Lib/pdb.py updateweights_tests.py

2. Function gud-pdb-marker-filter (gud.el) uses regular expression gud-pdb-marker-regexp to identify specific parts of the Pdb output. This regular expression is defined as follows:

(defvar gud-pdb-marker-regexp
"^> \\([-a-zA-Z0-9_/.:\\]*\\|\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|\\)()\\(->[^\n]*\\)?\n")

The first group of the regexp should match any file path. However, it fails to recognize paths that contain a space, such as the path that contained my sources. This was easily fixed by the addition of a space to the first group:

(defvar gud-pdb-marker-regexp
"^> \\([-a-zA-Z0-9_ /.:\\]*\\|\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|\\)()\\(->[^\n]*\\)?\n")

I use nosetests to automatically collect my unit tests and execute them. It is very easy to run nosetests from Emacs and automatically invoke Pdb as soon as a unittest.TestCase assertion fails. First, create a Python module that invokes nose.run():

import nose; nose.run()

Then M-x the command pdb and enter the following command-line in the minibuffer:

python -u nosetests.py --pdb-failures --nocapture --quiet

The options "--pdb-failures --nocapture --quiet" are intended for and automatically picked up by the call to Nose.run. The nosetests "Usage" message has the following to say about these options:

--pdb-failures Drop into debugger on failures
-s, --nocapture Don't capture stdout (any stdout output will be
printed immediately) [NOSE_NOCAPTURE]
-q, --quiet Be less verbose

The "--quiet" option is required so nose does not output messages that make it impossible for Emacs to parse the Pdb output.

3 comments:

Unknown said...

I am grateful for this posting! It solved my pdb-under-emacs-on-windows problem!

I feel that pdb-under-emacs is far superior than debugging/coding under IDLE (of course, IDLE generally "just works", which *is* pretty nice).

Anyway, you nailed it, and I can use my emacs again. thank you!

zacoder said...
This comment has been removed by the author.
Wenlong Feng said...

awesome!
Just curious how you find the deadlock issue, by debugging emacs source code?