IPython?

For me, this is the cat's pyjamas. IPython is an enhanced interactive Python shell that comes from the Scientific Python community. You'll never want to use the plain interpreter again.

Here's the simplest way to start up the debugger with IPython?:

jean@klippie work $ ./bin/zopectl debug
Starting debugger (the name "app" is bound to the top-level Zope object)
>>> import IPython
>>> IPython.Shell.IPShell().mainloop(sys_exit=1)
In [1]: @autocall # Otherwise IPython tries to *call* app
Automatic calling is: OFF
In [2]: app = __IP.internal_ns['app']

2006-10-03: Another way is to modify the debugzope script. Right after the last line (del Debugger) add this (same indentation as the last line):

import IPython
IPython.Shell.IPShell(user_ns=locals()).mainloop(sys_exit=1)

You must then set "autocall 0" in ~/.ipython/ipythonrc. And then runnning debugzope will drop you into an IPython?-shell with the app and root objects available.

I often add some common defines:

# Common dev stuff I'm using ..
from Acquisition import aq_inner, aq_parent, aq_base, aq_chain, aq_get
from Products.CMFCore.utils import getToolByName
membership_tool = getToolByName(app.fab, 'portal_membership')
# etc ..

Now I can autocomplete to my heart's content:

In [15]: app.acl_users.getU
app.acl_users.getUser                app.acl_users.getUserNames__roles__
app.acl_users.getUserById            app.acl_users.getUser__roles__
app.acl_users.getUserById__roles__   app.acl_users.getUsers
app.acl_users.getUserNames           app.acl_users.getUsers__roles__

In [15]: app.acl_users.getUser?? # Get docs as well as the source
Type:           Python Method
Base Class:     <type 'function'>
String Form:    <bound method UserFolder.getUser of <UserFolder instance at 411297d0>>
Namespace:      Interactive
File:           /home/jean/Zope-2.7.1-0/lib/python/AccessControl/User.py
Definition:     app.acl_users.getUser(self, name)
Source:
def getUser(self, name):
        """Return the named user object or None"""
        return self.data.get(name, None)

Debugging as an authenticated user (failing)

Fetch the user, and create a new security context for it. For our first try, I'm going to make an intentional mistake, and forget to wrap the user we're fetching:

In [17]: from AccessControl.SecurityManagement import newSecurityManager
In [18]: jean = app.acl_users.getUser('jean')
In [19]: newSecurityManager?
Type:           function
Base Class:     <type 'function'>
String Form:    <function newSecurityManager at 0x40bb7c34>
Namespace:      Interactive
File:           /home/jean/Zope-2.7.1-0/lib/python/AccessControl/SecurityManagement.py
Definition:     newSecurityManager(request, user)
Docstring:
    Set up a new security context for a request for a user
In [21]: newSecurityManager(jean)

Now, if I try to create some new content, I'll get an error due to the fact that my user object isn't wrapped in an acquisition context. IPython? shows me a detailed, colored traceback of which I'm quoting only the last few lines:

In [22]: mat = app.fab.materials.invokeFactory('Material', 'test')
AttributeError           Traceback (most recent call last)
...
    282     uid=user.getId()
    283     if uid is None: return uid
--> 284     db=user.aq_inner.aq_parent
    285     path=[absattr(db.id)]
    286     root=db.getPhysicalRoot()

AttributeError: aq_inner

In [23]: jean = jean.__of__(app.fab) # Let's try wrapping (wrongly!!)
In [24]: @pdb # Now IPython will drop us in a debugger upon exception
Automatic pdb calling has been turned ON
In [25]: mat = app.fab.materials.invokeFactory('Material', 'test1')
...
/home/jean/Zope-2.7.1-0/lib/python/AccessControl/Owned.py in getOwner(self, info, aq_get, UnownableOwner, getSecurityManager)
     92             user = SpecialUsers.nobody
     93         else:
---> 94             user = udb.getUserById(oid, None)
     95             if user is None: user = SpecialUsers.nobody
     96         return user

AttributeError: getUserById
> /home/jean/Zope-2.7.1-0/lib/python/AccessControl/Owned.py(94)getOwner()
-> user = udb.getUserById(oid, None)
(Pdb) udb
<PloneSite instance at 41124b30>
(Pdb) udb.getUserById
*** AttributeError: getUserById
(Pdb) udb.acl_users.getUserById
<bound method GroupUserFolder.getUserById of <GroupUserFolder instance at 41be9ad0>>

Debugging as an authenticated user (successfully)

Here I'm getting the user from the 'acl_users' in the root, and wrapping it in Plone's User Folder. Calling 'invokeFactory' works:

In [10]: from AccessControl.SecurityManagement import newSecurityManager, getSecurityManager
In [11]: user = app.acl_users.getUser('jean').__of__(app.portal.acl_users)
In [12]: newSecurityManager(None, user)
In [13]: getSecurityManager().getUser() # just to verify it.
Out[13]: jean
In [14]: app.fab.materials.invokeFactory('Material', 'test11')
In [15]: mat = getattr(app.fab.materials, 'test11')
In [16]: mat
Out[16]: <Material at /fab/materials/test11>

Now the object can be edited and so on:

In [17]: mat.edit(text_format='plain/html',text='dummy body')
In [18]: mat.setTitle('dummy title')

And I can commit the transaction to persist it in the ZODB:

In [19]: app._p_jar.sync()
In [20]: get_transaction().commit()

Debugging Zope 3 dependent code

With Plone 3.0 lots of stuff is zope 3 dependent. Normally this happens in traversal, but needs to be done manually at a debug prompt. Thanks to Martin Aspeli for this tip.:

In [21]: from zope.app.component.hooks import setSite
In [22]: setSite(app.portal)

Connecting to the monitor port

You can connect to a monitor port of the medusa server. First, in 'zope.conf' you need to tell Zope to start a monitor server:

<monitor-server>
  # valid keys are "address"
  address 9082
</monitor-server>

Then, you can connect to it like so, and get a Python prompt:

jean@klippie work $ python ~/Zope-2.7.1-0/lib/python/ZServer/medusa_client.py localhost 9082

Enter Password:
Welcome to <socket._socketobject object at 0x41ab1c84>
>>>

The password is the emergency user's password, defined in the 'access' file in the root of your Zope instance.

To be honest, I have no idea what you can do at the monitor prompt. I haven't used it.

Debugging the results of calling an URL through the web

(This section lifted from mcdonc's Howto )

If a page template raises an exception, you can do this:

In [10]: import Zope
In [11]: Zope.debug('/path/to/object?query_string', u='username:password', pm=1)
.... output will happen
In [12]: import pdb
In [13]: pdb.pm()

You will be placed at the point where the exception occured in the code. You can't step through code from here, but you can examine the values of the code where the error occured. Note the usefulness of the "u=" parameter which lets you set your user to any user id without needing a through-the-web authentication.

To do normal debugging, instead of the "pm=1" argument, provide an argument to Zope.debug of "d=1". This will allow you to step through code ala Michel's HowTo?.

Alternative shell script from Martijn Pieters

From a mail by Martijn:

I use the attached shellscript to set up the ipython shell with the same environment as what you'd get when you execute zopectl debug.

The attached version is for use with python 2.4 (Zope 2.9 and up), but it should work fine with python 2.3 and Zope 2.8 as well if you rename the ipython command on the last line. For Zope 2.7 and lower, you'll need to rename both occurences of 'Zope2' with 'Zope' as well.

Here's the script:

#!/bin/sh
# Zope debug prompt for IPython
set -e
# Assume instance home is cwd
INSTANCE_HOME=`pwd`
# unless we specified a path on the command line
[ $# -ne 0 ] && INSTANCE_HOME="$@"

# Get Zope paths from the zope configuration file
ZOPE_CONFIG=$INSTANCE_HOME/etc/zope.conf
ZOPE_HOME=`grep '%define ZOPE' $ZOPE_CONFIG | cut -d ' ' -f 3`
SOFTWARE_HOME=$ZOPE_HOME/lib/python
PYTHONPATH="$SOFTWARE_HOME:$INSTANCE_HOME/lib/python"

export PYTHONPATH ZOPE_CONFIG SOFTWARE_HOME INSTANCE_HOME

# Set up Zope within IPython
# Imports
STARTUP="import __builtin__, sys"
# Zope wants to mark 'quit' as safe for scripting, but IPython lacks it
# Create a dummy
STARTUP="$STARTUP; setattr(__builtin__, 'quit', None)"
# Import Zope and store the database root in the app variable
STARTUP="$STARTUP; import Zope2"
STARTUP="$STARTUP; app=Zope2.app()"
# Make sure stdin is correctly pointing to the default, otherwise readline is
# borked. At least WingDBG will redirect it, possibly others
STARTUP="$STARTUP; sys.stdin = sys.__stdin__"
# Remove our imports
STARTUP="$STARTUP; del __builtin__, sys"
# And print a banner
MESSAGE="Starting debugger (the name \"app\" is bound to the top-level Zope object)"
STARTUP="$STARTUP; __IP.write('$MESSAGE')"

/usr/bin/python2.4-ipython -c "$STARTUP"

Acknowledgements

Without lots of help from the mailing lists and web, this wouldn't exist. The first examples above come from Jens Vagelpohl, Paul Winkler, and J Cameron Cooper.