Chapter 22
Object to File System mapping using FTP as example

Difficulty

Newcomer

Skills

Problem/Task

Zope provides by default an FTP server , which is a filesystem based protocol. This immediately raises the question about the way objects are mapped to the filesystem representation and back. To accomplish the mapping in a flexible and exchangeable way, there exists a set of interfaces that can be implemented as adapters to provide a representation that the FTP Publisher understands. This chapter shows how to implement some of the interfaces for a custom filesystem representation.

Solution

At this point you might wonder: “Why in the world would we have to write our own filesystem support? Is Zope not providing any implementations by default?” Well, to answer the latter question: Yes and no. There is an adapter registered for IContainer to IReadDirectory and IWriteDirectory. However, they are not very useful, since our Message Board and Message objects are not only containers but also contain content that is not displayed anywhere. Just start Zope 3 and check it out yourself. Thus it will be the goal of this chapter to create a representation that handles the containment and the content at the same time.

Since the core has already a lot of code presenting IContainer implementations as directories, we should reuse this part of the framework. The content of an object could be simply represented by a virtual file called contents, which contains all relevant data normalized as simple plain text. Note also that we will not need to have a complete mapping between the object and the filesystem representation, since we do not need or want to expose annotations for example. I suggest that the contents of the MessageBoard object simply contains the data of the description attribute and for the Message I propose the following format:


1  Title: <message title>
2  
3  <message body>

This way we can also parse easily the title when new contents is submitted, since we want to implement the read and write interfaces of the filesystem representation. One of the main goals is to keep the VirtualContentsFile class as generic as possible, so that we can use it for both message boards and messages. To do that the virtual file must delegate the request to create the plain text representation to a component that knows about the specifics of the respective content object. For this task, we will have a simple IPlainText adapter that provides the plain text representation of each content component’s contents.

22.1 Step I: Plain Text Adapters

As said above, IPlainText adapters are responsible to return and accept the plain text representation of the object’s content and just do the right thing with the data. They are very simple objects, having two methods, one for providing and one for processing the text.

22.1.1 (a) The Interface

The interface is simple, it defines a getText() and a setText() method:


1  class IPlainText(Interface):
2      """This interface allows you to represent an object's content in plain
3      text."""
4  
5      def getText():
6          """Get a pure text representation of the object's content."""
7  
8      def setText(text):
9          """Write the text to the object."""

This interface should be placed in the interfaces module of the messageboard. In the doc strings I refer to the object’s “content” without further qualifying it. With content I mean all data (property values) that should be represented via the plain text representation.

The setText() method could in theory become quite complex, depending on how many properties you want to map and how you represent them. You will see that in our two cases it will be still fairly simple.

22.1.2 (b) The Implementation

We need two implementations, one for the Message and one for the MessageBoard class. Note that I skipped writing tests at this point, since the functionality of these adapters are carefully tested in the following code.

First, we write the message board adapter, so open the messageboard.py file and add the following code:


1  from book.messageboard.interfaces import IPlainText
2  
3  class PlainText:
4  
5      implements(IPlainText)
6  
7      def __init__(self, context):
8          self.context = context
9  
10      def getText(self):
11          return self.context.description
12  
13      def setText(self, text):
14          self.context.description = unicode(text)

This is an extremely simple implementation of the IPlainText interface, since we map only one attribute.

The implementation for the Message (put it in message.py) looks like this:


1  from book.messageboard.interfaces import IPlainText
2  
3  class PlainText:
4  
5      implements(IPlainText)
6  
7      def __init__(self, context):
8          self.context = context
9  
10      def getText(self):
11          return 'Title: %s\n\n%s' %(self.context.title,
12                                     self.context.body)
13  
14      def setText(self, text):
15          if text.startswith('Title: '):
16              title, text = text.split('\n', 1)
17              self.context.title = title[7:]
18  
19          self.context.body = text.strip()

This implementation is more interesting, since we map two properties to the plain text.

22.1.3 (c) The Configuration

The last step is to register the two adapters with the Component Architecture. Just add the following two adapter directives to configure.zcml:


1  <adapter
2      factory=".messageboard.PlainText"
3      provides=".interfaces.IPlainText"
4      for=".interfaces.IMessageBoard" />
5  
6  <adapter
7      factory=".message.PlainText"
8      provides=".interfaces.IPlainText"
9      for=".interfaces.IMessage" />

We are now done. Even though we have two new fully-functional components at this point, we gained no new functionality yet, since these adapters are not used anywhere. We have to complete the next two sections to see any results.

22.2 Step II: The “Virtual Contents File” Adapter

How we implement the virtual contents file is fully up to us. However, there are benefits of choosing one way over another, since it will save us some work. The best method is to create a new interface IVirtualContentsFile, which extends zope.app.file.interfaces.IFile. The advantage is that there are already filesystem-specific adapters (implementing zope.app.filerepresentation.interfaces.IReadFile and zope.app.filerepresentation.interfaces.IWriteFile) for the above mentioned interface. IFile might not be the best and most concise interface for our needs, but the advantages of using it are very convincing.

22.2.1 (a) The Interface

When you look through the Zope 3 source code, you will notice that the IFile and IFileContent interfaces go hand in hand with each. Thus, our virtual contents file interface will extend both of these interfaces.


1  from zope.app.file.interfaces import IFile, IFileContent
2  
3  class IVirtualContentsFile(IFile, IFileContent):
4      """Marker Interface to mark special Message and Message Board
5      Contents files in FS representations."""

22.2.2 (b) The Implementation

Now the fun begins. First we note that IFile requires three properties, contentType, data, and size. While data and size are obvious, we need to think a bit about contentType. Since we really just want to return always text/plain, the accessor should statically return text/plain and the mutator should just ignore the input.

To make a long story short, here is the code, which you should place in a new file called filerepresentation.py:


1  from zope.interface import implements
2  from interfaces import IVirtualContentsFile, IPlainText
3  
4  class VirtualContentsFile(object):
5  
6      implements(IVirtualContentsFile)
7  
8      def __init__(self, context):
9          self.context = context
10  
11      def setContentType(self, contentType):
12          '''See interface IFile'''
13          pass
14  
15      def getContentType(self):
16          '''See interface IFile'''
17          return u'text/plain'
18  
19      contentType = property(getContentType, setContentType)
20  
21      def edit(self, data, contentType=None):
22          '''See interface IFile'''
23          self.setData(data)
24  
25      def getData(self):
26          '''See interface IFile'''
27          adapter = IPlainText(self.context)
28          return adapter.getText() or u''
29  
30      def setData(self, data):
31          '''See interface IFile'''
32          adapter = IPlainText(self.context)
33          return adapter.setText(data)
34  
35      data = property(getData, setData)
36  
37      def getSize(self):
38          '''See interface IFile'''
39          return len(self.getData())
40  
41      size = property(getSize)

This was pretty straightforward. There are really no surprises here.

22.2.3 (c) The Tests

Since even the last coding step did not provide a functional piece of code, it becomes so much more important to write some careful tests for the VirtualContentsFile component. Another requirement of the tests are that this adapter is being tested with both MessageBoard and Message instances. To realize this, we write an a base test and then realize this test for each component. So in the tests folder, create a new file called test_filerepresentation.py and add the following content:


1  import unittest
2  from zope.interface.verify import verifyObject
3  from zope.app import zapi
4  from zope.app.tests import ztapi
5  from zope.app.tests.placelesssetup import PlacelessSetup
6  
7  from book.messageboard.interfaces import \
8       IVirtualContentsFile, IPlainText, IMessage, IMessageBoard
9    from book.messageboard.message import \
10       Message, PlainText as MessagePlainText
11  from book.messageboard.messageboard import \
12       MessageBoard, PlainText as MessageBoardPlainText
13  from book.messageboard.filerepresentation import VirtualContentsFile
14  
15  class VirtualContentsFileTestBase(PlacelessSetup):
16  
17      def _makeFile(self):
18          raise NotImplemented
19  
20      def _registerPlainTextAdapter(self):
21          raise NotImplemented
22  
23      def setUp(self):
24          PlacelessSetup.setUp(self)
25          self._registerPlainTextAdapter()
26  
27      def testContentType(self):
28          file = self._makeFile()
29          self.assertEqual(file.getContentType(), 'text/plain')
30          file.setContentType('text/html')
31          self.assertEqual(file.getContentType(), 'text/plain')
32          self.assertEqual(file.contentType, 'text/plain')
33  
34      def testData(self):
35          file = self._makeFile()
36  
37          file.setData('Foobar')
38          self.assert_(file.getData().find('Foobar') >= 0)
39          self.assert_(file.data.find('Foobar') >= 0)
40  
41          file.edit('Blah', 'text/html')
42          self.assertEqual(file.contentType, 'text/plain')
43          self.assert_(file.data.find('Blah') >= 0)
44  
45      def testInterface(self):
46          file = self._makeFile()
47          self.failUnless(IVirtualContentsFile.providedBy(file))
48          self.failUnless(verifyObject(IVirtualContentsFile, file))
49  
50  
51  class MessageVirtualContentsFileTest(VirtualContentsFileTestBase,
52                                       unittest.TestCase):
53  
54      def _makeFile(self):
55          return VirtualContentsFile(Message())
56  
57      def _registerPlainTextAdapter(self):
58          ztapi.provideAdapter(IMessage, IPlainText, MessagePlainText)
59  
60      def testMessageSpecifics(self):
61          file = self._makeFile()
62          self.assertEqual(file.context.title, '')
63          self.assertEqual(file.context.body, '')
64          file.data = 'Title: Hello\n\nWorld'
65          self.assertEqual(file.context.title, 'Hello')
66          self.assertEqual(file.context.body, 'World')
67          file.data = 'World 2'
68          self.assertEqual(file.context.body, 'World 2')
69  
70  
71  class MessageBoardVirtualContentsFileTest(
72        VirtualContentsFileTestBase, unittest.TestCase):
73  
74      def _makeFile(self):
75          return VirtualContentsFile(MessageBoard())
76  
77      def _registerPlainTextAdapter(self):
78          ztapi.provideAdapter(IMessageBoard, IPlainText,
79                               MessageBoardPlainText)
80  
81      def testMessageBoardSpecifics(self):
82          file = self._makeFile()
83          self.assertEqual(file.context.description, '')
84          file.data = 'Title: Hello\n\nWorld'
85          self.assertEqual(file.context.description,
86                           'Title: Hello\n\nWorld')
87          file.data = 'World 2'
88          self.assertEqual(file.context.description, 'World 2')
89  
90  def test_suite():
91      return unittest.TestSuite((
92          unittest.makeSuite(MessageVirtualContentsFileTest),
93          unittest.makeSuite(MessageBoardVirtualContentsFileTest),
94          ))
95  
96  if __name__ == '__main__':
97      unittest.main(defaultTest='test_suite')

You can now run the test and verify the functionality of the new tests.

22.2.4 (d) The Configuration

This section would not be complete without a registration. While we do not need to register the file representation component, we are required to make some security assertions about the object’s methods and properties. I simply copied the following security assertions from the File content component’s configuration.


1  <content class=".filerepresentation.VirtualContentsFile">
2  
3    <implements interface="
4        zope.app.annotation.interfaces.IAttributeAnnotatable" />
5  
6    <require
7        permission="book.messageboard.View"
8        interface="zope.app.filerepresentation.interfaces.IReadFile" />
9  
10    <require
11        permission="zope.messageboard.Edit"
12        interface="zope.app.filerepresentation.interfaces.IWriteFile"
13        set_schema="zope.app.filerepresentation.interfaces.IReadFile" />
14  
15  </content>

22.3 Step III: The IReadDirectory implementation

After all the preparations are complete, we are finally ready to give our content components, MessageBoard and Message, a cool filesystem representation.

22.3.1 (a) The Implementation

The first fact we should notice is that zope.app.filerepresentation.ReadDirectory has already a nice implementation, except for the superfluous SiteManager and the missing contents file. So we simply take this class (subclass it) and merely overwrite keys(), get(key,default=None), and __len__(). All other methods depend on these three. So our code for the ReadDirectory looks something like that (place in filerepresentation.py):


1  from zope.app.filerepresentation.interfaces import IReadDirectory
2  from zope.app.folder.filerepresentation import \
3       ReadDirectory as ReadDirectoryBase
4  
5  class ReadDirectory(ReadDirectoryBase):
6      """An special implementation of the directory."""
7  
8      implements(IReadDirectory)
9  
10      def keys(self):
11          keys = self.context.keys()
12          return list(keys) + ['contents']
13  
14      def get(self, key, default=None):
15          if key == 'contents':
16              return VirtualContentsFile(self.context)
17          return self.context.get(key, default)
18  
19      def __len__(self):
20          l = len(self.context)
21          return l+1

Now we are done with our implementation. Let’s write some unit tests to ensure the functionality and then register the filesystem components.

22.3.2 (b) The Tests

For testing the ReadDirectory implementation, we again need to test it with the MessageBoard and Message components. So similar to the previous tests, we have a base test with specific implementations. Also note that it will not be necessary to test all IReadDirectory methods, since they are already tested in the base class tests. So we are just going to test the methods we have overridden:


1  from book.messageboard.filerepresentation import ReadDirectory
2  
3  class ReadDirectoryTestBase(PlacelessSetup):
4  
5      def _makeDirectoryObject(self):
6          raise NotImplemented
7  
8      def _makeTree(self):
9          base = self._makeDirectoryObject()
10          msg1 = Message()
11          msg1.title = 'Message 1'
12          msg1.description = 'This is Message 1.'
13          msg11 = Message()
14          msg11.title = 'Message 1-1'
15          msg11.description = 'This is Message 1-1.'
16          msg2 = Message()
17          msg2.title = 'Message 1'
18          msg2.description = 'This is Message 1.'
19          msg1['msg11'] = msg11
20          base['msg1'] = msg1
21          base['msg2'] = msg2
22          return ReadDirectory(base)
23  
24      def testKeys(self):
25          tree = self._makeTree()
26          keys = list(tree.keys())
27          keys.sort()
28          self.assertEqual(keys, ['contents', 'msg1', 'msg2'])
29          keys = list(ReadDirectory(tree['msg1']).keys())
30          keys.sort()
31          self.assertEqual(keys, ['contents', 'msg11'])
32  
33      def testGet(self):
34          tree = self._makeTree()
35          self.assertEqual(tree.get('msg1'), tree.context['msg1'])
36          self.assertEqual(tree.get('msg3'), None)
37          default = object()
38          self.assertEqual(tree.get('msg3', default), default)
39          self.assertEqual(tree.get('contents').__class__,
40                           VirtualContentsFile)
41  
42      def testLen(self):
43          tree = self._makeTree()
44          self.assertEqual(len(tree), 3)
45          self.assertEqual(len(ReadDirectory(tree['msg1'])), 2)
46          self.assertEqual(len(ReadDirectory(tree['msg2'])), 1)
47  
48  
49  class MessageReadDirectoryTest(ReadDirectoryTestBase,
50                                 unittest.TestCase):
51  
52      def _makeDirectoryObject(self):
53          return Message()
54  
55  
56  class MessageBoardReadDirectoryTest(ReadDirectoryTestBase,
57                                      unittest.TestCase):
58  
59      def _makeDirectoryObject(self):
60          return MessageBoard()

After you are done writing the tests, do not forget to add the two new TestCases to the TestSuite.

22.3.3 (c) The Configuration

Finally we simply register our new components properly using the following ZCML directives:


1  <adapter
2     for=".interfaces.IMessageBoard"
3     provides="zope.app.filerepresentation.interfaces.IReadDirectory"
4     factory=".filerepresentation.ReadDirectory"
5     permission="zope.View"/>
6  
7  <adapter
8     for=".interfaces.IMessage"
9     provides="zope.app.filerepresentation.interfaces.IReadDirectory"
10     factory=".filerepresentation.ReadDirectory"
11     permission="zope.View"/>

That’s it. You can now restart Zope and test the filesystem representation with an FTP client of your choice. In the following sequence diagram you can see how a request is guided to find its information and return it properly.

PIC

Figure 22.1: Collaboration diagram of the inner working from requesting the contents “file” to receiving the actual data.

22.4 Step IV: The Icing on the Cake - A special Directory Factory

While you were playing around with the new filesystem support, you might have tried to create a directory to see what happened and it probably just caused a system error, since no adapter was found from IMessage/ IMessageBoard to IDirectoryFactory. Since such a behavior is undesirable, we should create a custom adapter that provides IDirectoryFactory. The IWriteDirectory adapter of any container object then knows how to make use of this factory adapter. So we add the following trivial factory to our filesystem code:


1  from zope.app.filerepresentation.interfaces import IDirectoryFactory
2  from message import Message
3  
4  class MessageFactory(object):
5      """A simple message factory for file system representations."""
6  
7      implements(IDirectoryFactory)
8  
9      def __init__(self, context):
10          self.context = context
11  
12      def __call__(self, name):
13          """See IDirectoryFactory interface."""
14          return Message()

Registering the factory is just a matter of two adapter directives (one for each content component):


1  <adapter
2     for=".interfaces.IMessageBoard"
3     provides="zope.app.filerepresentation.interfaces.IDirectoryFactory"
4     factory=".filerepresentation.MessageFactory"
5     permission="zope.View" />
6  
7  <adapter
8     for=".interfaces.IMessage"
9     provides="zope.app.filerepresentation.interfaces.IDirectoryFactory"
10     factory=".filerepresentation.MessageFactory"
11     permission="zope.View" />

Now we have finally made it. The filesystem support should be very smooth and usable at this point. You should be able to view all relevant content, upload new contents data and create new messages. The only problem that might remain is that some FTP clients (like KDE’s FTP support) try to upload the contents file as contents.part and then rename it to contents. Since our filesystem code does not support such a feature, this will cause an error; see exercise 2 for details.

Exercises

  1. Currently there is no creation/modification date/time or creator defined for the virtual contents file. This is due to the fact that the respective Dublin Core properties were not set. The virtual file should really receive the same Dublin Core the MessageBoard or Message has. The easiest way would be simply to copy the Dublin Core annotation. Do that and make sure the data and the user are shown correctly.
  2. In the final remarks of this chapter I stated that the current code will not work very well with some clients, such as Konqueror. Fix the code to support this behavior. The best would be to store the temporary file in an annotation and delete it, once it was moved.