Chapter 35
Changing Traversal Behavior

Difficulty

Sprinter

Skills

Be familiar with the Zope 3 component architecture and testing framework.
Some knowledge about the container interfaces is helpful. Optional.

Problem/Task

Zope 3 uses a mechanism called “traversal” to resolve an object path, as given by a URL, to the actual object. Obviously, there is some policy involved in the traversal process, as objects must be found, namespaces must be resolved, and even components, such as views, be looked up in the component architecture. This also means that these policies can be changed and replaced. This chapter will show you how to change the traversal policy, so that the container items are not case-sensitive anymore.

Solution

In Zope 3, traversers, objects that are responsible for using a path segment to get from one object to another, are just implemented as views of the respective presentation type. In principle the traverser only has to implement IPublishTraverse (located in zope.publisher.interfaces), which specifies a method named publishTraverse(request,name) that returns the traversed object. The browser implementation, for example, is simply a view that tries to resolve name using its context. Whether the method tries to access sub-objects or look up views called name is up to the specific implementation, like the zope.app.container.traversal.ContainerTraverser for the browser.

35.1 Step I: The Case-Insensitive Folder

As mentioned before, in this chapter we are going to implement a case-insensitive traverser and a sample folder that uses this traverser called CaseInsensitiveFolder. Let’s develop the latter component first. All we need for the case-insensitive folder is an interface and a factory that provides the for a normal Folder instance.

Create a package called insensitivefolder in the book package. In the __init__.py file add the following interface and factory:

1  from zope.component.interfaces import IFactory
2  from zope.app.folder import Folder
3  from zope.app.folder.interfaces import IFolder
4  from zope.interface import implements, implementedBy
5  from zope.interface import directlyProvides, directlyProvidedBy
6
7  class ICaseInsensitiveFolder(IFolder):
8      """Marker for folders whose contained items keys are case insensitive.
9
10      When traversing in this folder, all names will be converted to lower
11      case. For example, if the traverser requests an item called `Foo`, in
12      reality the first item matching `foo` or any upper-and-lowercase
13      variants are looked up in the container."""
14
15  class CaseInsensitiveFolderFactory(object):
16      """A Factory that creates case-insensitive Folders."""
17      implements(IFactory)
18
19      def __call__(self):
20          """See zope.component.interfaces.IFactory
21
22          Create a folder and mark it as case insensitive.
23          """
24          folder = Folder()
25          directlyProvides(folder, directlyProvidedBy(folder),
26                           ICaseInsensitiveFolder)
27          return folder
28
29      def getInterfaces(self):
30          """See zope.component.interfaces.IFactory"""
31          return implementedBy(Folder) + ICaseInsensitiveFolder
32
33  caseInsensitiveFolderFactory = CaseInsensitiveFolderFactory()

Instead of developing a new content type, we create a factory that tags on the marker interface making the Folder instance an ICaseInsensitiveFolder. This is a classic example of declaring and using a factory directly. Factories are used in many places, but usually they are auto-generated in ZCML handlers.

The factory is simply registered in ZCML using

1  <configure
2      xmlns="http://namespaces.zope.org/zope"
3      xmlns:browser="http://namespaces.zope.org/browser"
4      i18n_domain="zope">
5
6  <factory
7      id="zope.CaseInsensitiveFolder"
8      component=".caseInsensitiveFolderFactory"
9      />
10
11  <browser:addMenuItem
12      factory="zope.CaseInsensitiveFolder"
13      title="Case insensitive Folder"
14      description="A simple case insensitive Folder."
15      permission="zope.ManageContent"
16      />
17
18  <browser:icon
19      name="zmi_icon"
20      for=".ICaseInsensitiveFolder"
21      file="cifolder_icon.png"
22      />
23
24  </configure>

Line 6-9: Declare the factory. The id must be a valid Id field value.
Line 11-16: Declare an add menu item entry using the factory id, as specified in the id attribute before.
Line 18-22: Register also a custom icon for the case-insensitive folder, so that we can differentiate it from the other folders. The icon can be found in the repository.

35.2 Step II: The Traverser

Now we have a new content type, but it does not do anything special. We have to implement the special traverser for the case-insensitive folder. Luckily, we do not have to implement a new container traverser from scratch, but just use the standard ContainerTraverser and replace the publishTraverse() method to be a bit more flexible and ignore the case of the item names. In the __init__.py file add the following traverser class:

1  from zope.publisher.interfaces import NotFound
2
3  from zope.app import zapi
4  from zope.app.container.traversal import ContainerTraverser
5
6  class CaseInsensitiveFolderTraverser(ContainerTraverser):
7
8      __used_for__ = ICaseInsensitiveFolder
9
10      def publishTraverse(self, request, name):
11          """See zope.publisher.interfaces.browser.IBrowserPublisher"""
12          subob = self._guessTraverse(name)
13          if subob is None:
14              view = zapi.queryView(self.context, name, request)
15              if view is not None:
16                  return view
17
18              raise NotFound(self.context, name, request)
19
20          return subob
21
22      def _guessTraverse(self, name):
23          for key in self.context.keys():
24              if key.lower() == name.lower():
25                  return self.context[key]
26          return None

Line 8: Just as information, this traverser is only meant to be used with ICaseInsensitiveFolder components. However, the following code is generic enough that it would work with any object implementing IReadContainer. Note that the most generic container traverser is registered for ISimpleReadContainer, which is not sufficient here, since we make use of the keys() method, which is not available in ISimpleReadContainer.
Line 10-20: First we try to find the name using the private _ guessTraverse() method. If no object was found in the items of the container, we check whether name could be a view and return it. If the name does not point to an item or a view, then we need to raise a NotFound error.
Note that the implementation of this method could have been more efficient. We could first try to get the object using _guessTraverse() and upon failure forward the request to the original publishTraverse() method of the base class. Then the code would look like this:

1  def publishTraverse(self, request, name):
2      subob = self._guessTraverse(name)
3      if subob is not None:
4          return subob
5      return super(CaseInsensitiveFolderTraverser,
6                   self).publishTraverse(request, name)

However, this would have hidden some of the insights on how publishTraverse() should behave.
Line 22-26: Here we try to look up the name without caring about the case. This works both ways. The keys of the container and the provided name are converted to all lower case. We then compare the two. If a match is found, the value for the key is returned. Note that we need to keep the original key (having upper and lower case), since the container still manages the keys in a case-sensitive manner.

The traverser is registered via ZCML simply using the zope:view directive:

1  <view
2      for=".ICaseInsensitiveFolder"
3      type="zope.publisher.interfaces.browser.IBrowserRequest"
4      factory=".CaseInsensitiveFolderTraverser"
5      provides="zope.publisher.interfaces.browser.IBrowserPublisher"
6      permission="zope.Public"
7      />

Line 2: Register the view only for case-insensitive folders.
Line 3: Make sure that this traverser is only used for browser requests.
Line 4: It is very important to specify the provided interface here, so that we know that the object is a browser publisher and implements the sufficient interfaces for traversal.
Line 5: We want to allow everyone to be able to traverse through the folder, since it does not you anyone any special access. All methods of the returned object are protected separately anyways.

To register the product with Zope 3, add a file named insensitivefolder-configure.zcml to package-includes. It should contain the following line:

1 <include package="book.insensitivefolder" />

Once you restart Zope 3, the new folder should be available to you.

35.3 Step III: Unit Tests

Unit tests can be quickly written, since we can make use of the original container traverser’s unit tests and setup. Open a tests.py file to insert the following test code.

1  import unittest
2  from zope.app.container.tests import test_containertraverser
3  from book.insensitivefolder import CaseInsensitiveFolderTraverser
4
5  class Container(test_containertraverser.TestContainer):
6
7      def keys(self):
8          return self.__dict__.keys()
9
10      def __getitem__(self, name):
11          return self.__dict__[name]
12
13  class InsensitiveCaseTraverserTest(test_containertraverser.TraverserTest):
14
15      def _getTraverser(self, context, request):
16          return CaseInsensitiveFolderTraverser(context, request)
17
18      def _getContainer(self, **kw):
19          return Container(**kw)
20
21      def test_allLowerCaseItemTraversal(self):
22          self.assertEquals(
23              self.traverser.publishTraverse(self.request, 'foo'),
24              self.foo)
25          self.assertEquals(
26              self.traverser.publishTraverse(self.request, 'foO'),
27              self.foo)
28
29  def test_suite():
30      return unittest.TestSuite((
31          unittest.makeSuite(InsensitiveCaseTraverserTest),
32          ))
33
34  if __name__ == '__main__':
35      unittest.main(defaultTest='test_suite')

Line 7-11: The original test container has to be extended to support keys() and __getitem__(), since both of these methods are need for the case-insensitive traverser.
Line 15-16: A setup helper method that returns the traverser. This method is used in the setUp() method to construct the environment.
Line 18-19: Another helper method that allows us to specify a custom container. This method is used in the setUp() method to construct the environment.
Line 21-27: Most of the functionality is already tested in the base test case. Here we just test that the case of the letters is truly ignored.
Line 29-35: This is the usual unit test boilerplate.

As always, the tests are directly executable, once Zope 3 is in your path.

35.4 Step IV: Functional Tests

Before we let the code be run in the wild, we should test it some more in a fairly contained environment. The following functional test is pretty straight forward and mimics the unit tests.

Open a file called ftests.py and add

1  import unittest
2  from zope.app.tests.functional import BrowserTestCase
3  from zope.publisher.interfaces import NotFound
4
5  class TestCaseInsensitiveFolder(BrowserTestCase):
6
7      def testAddCaseInsensitiveFolder(self):
8          # Step 1: add the case insensitive folder
9          response = self.publish(
10              '/+/action.html',
11              basic='mgr:mgrpw',
12              form={'type_name': book.CaseInsensitiveFolder',
13                    'id': u'cisf'})
14          self.assertEqual(response.getStatus(), 302)
15          self.assertEqual(response.getHeader('Location'),
16                           'http://localhost/@@contents.html')
17          # Step 2: add the file
18          response = self.publish('/cisf/+/action.html',
19                                  basic='mgr:mgrpw',
20                                  form={'type_name': u'zope.app.content.File',
21                                        'id': u'foo'})
22          self.assertEqual(response.getStatus(), 302)
23          self.assertEqual(response.getHeader('Location'),
24                           'http://localhost/cisf/@@contents.html')
25          # Step 3: check that the file is traversed
26          response = self.publish('/cisf/foo')
27          self.assertEqual(response.getStatus(), 200)
28          response = self.publish('/cisf/foO')
29          self.assertEqual(response.getStatus(), 200)
30          self.assertRaises(NotFound, self.publish, '/cisf/bar')
31
32
33  def test_suite():
34      return unittest.TestSuite((
35          unittest.makeSuite(TestCaseInsensitiveFolder),
36          ))
37
38  if __name__ == '__main__':
39      unittest.main(defaultTest='test_suite')

There is really nothing interesting about this test. If you are not familiar with functional tests, read the corresponding chapter in the “Writing Tests” part of the book.

In Zope 2 it was common to change the traversal behavior of objects and containerish objects. In Zope 3, however, you will not need to implement your own traversers, since most of the time it is better and easier to write a custom IReadContainer content component.

The complete code of this product can be found at book/insensitivefolder. It was originally written by Vincenzo Di Somma and has been maintained by many developers throughout the development of Zope 3.

Exercises

The current implementation will only allow case-insensitive lookups through browser requests. But what about
1. XML-RPC and WebDAV, and
2. FTP?
Extend the existing implementations to support these protocols as well.
You might have already noticed that the implementation of this traverser is not quiet perfect and it might actually behave differently on various systems. Let’s say you have an object called “Foo” and “foo”. The way _guessTraverse() is implemented, it will use the key that is listed first in the list returned by keys(). However, the order the keys returned varies from system to system. Fix the behavior so that the behavior (whatever you decide it to be) is the same on every system. (Hint: This is a two line fix.)
It might be desirable, to change the policy of the name lookup a bit by giving exact key matches preference to case-insensitive matches. Change the traverser to implement this new policy.

Chapter 35Changing Traversal Behavior