Chapter 45
Writing Tests against Interfaces
Difficulty
Newcomer
Skills
- You should be familiar with Python interfaces. If necessary, read the “An
Introduction to Interfaces” chapter.
- You should know about the unittest package, especially the material
covered in the “Writing Basic Unit Tests” chapter.
Problem/Task
When one expects an interface to be implemented multiple times, it is good to
provide a set of generic tests that verify the correct semantics of this interface. In
Zope 3 we refer to these abstract tests as “interface tests”. This chapter will describe
how to implement and use such tests using two different implementations of a simple
interface.
Solution
45.1 Introduction
In Zope 3 we have many interfaces that we expect to be implemented multiple times.
The prime example is the IContainer interface, which is primarily implemented by
Folder, but also by many other objects that contain some other content. Here it
would be useful to implement some set of tests which verify that Folder and other
classes correctly implement IContainer.
Interface tests are abstract tests - i.e. they do not run by themselves, since they
do not know about any implementation - that provide a set of common tests. The
advantage of these tests is that the implementor of this interface immediately has
some feedback about his implementation of the interface. However, one should not
mistake the interface tests to be a replacement of a complete set of unit tests, but
rather as a supplement. Interface tests by definition cannot test implementation
details, something that is required of unit tests. Additional tests also ensure a higher
quality of code.
There are a couple of characteristics that you will be able to recognize in
any interface test. First, an interface should always have a test that verifies
that the interface implementation fulfills the contract. This can be done
using the verifyObject(interface,instance) method, which is found at
zope.interface.verify. Second, while the interface test is abstract, it needs to get
an instance of the implementation from somewhere. For this reason an interface test
should always provide an abstract method that returns an instance of the object. By
convention this method is called makeTestObject and it should look like
that:
1 def makeTestObject(self):
2 raise NotImplemented()
Each test case that inherits from the interface test should then realize this
method by returning an instance of the object to be tested.
But how can we determine what should be part of an interface test? The
best way to approach the problem is by thinking about the functionality
that the attributes and methods of the interface provide. You may also ask
about its behavior inside the system? Interface tests often model actual
usages of an object, while implementation tests also cover a lot of corner
cases and exceptions, something that is often hard to do with interface tests,
since you are bound to the interface-declared methods and attributes. From
another point of view, since tests should document an object, think of interface
tests as documentation on how the interface should be used and behave
normally.
45.2 The ISample Interface, Its Tests, and Its Implementations
Reusing the examples from the unit and doc test chapters, we develop an
ISample interface that provides a title attribute and uses the methods
getDescription and setDescription for dealing with the description of the
object.
Again, we would like to keep the code contained in one file for simplicity, so
open a file test_isample.py anywhere and add the following interface to
it:
1 from zope.interface import implements, Interface, Attribute
2
3 class ISample(Interface):
4 """This is a Sample."""
5
6 title = Attribute('The title of the sample')
7
8 def setDescription(value):
9 """Set the description of the Sample.
10
11 Only regular and unicode values should be accepted.
12 """
13
14 def getDescription():
15 """Return the value of the description."""
I assume you know about interfaces, so there is nothing interesting here. The next
step is to write the interface tests, so add the following TestCase class. You will
notice how similar these tests are to the ones developed before.
1 import unittest
2 from zope.interface.verify import verifyObject
3
4 class TestISample(unittest.TestCase):
5 """Test the ISample interface"""
6
7 def makeTestObject(self):
8 """Returns an ISample instance"""
9 raise NotImplemented()
10
11 def test_verifyInterfaceImplementation(self):
12 self.assert_(verifyObject(ISample, self.makeTestObject()))
13
14 def test_title(self):
15 sample = self.makeTestObject()
16 self.assertEqual(sample.title, None)
17 sample.title = 'Sample Title'
18 self.assertEqual(sample.title, 'Sample Title')
19
20 def test_setgetDescription(self):
21 sample = self.makeTestObject()
22 self.assertEqual(sample.getDescription(), '')
23 sample.setDescription('Description')
24 self.assertEqual(sample.getDescription(), 'Description')
25 self.assertRaises(AssertionError, sample.setDescription, None)
- Line 4-6: Here is the promised method to create object instances.
- Line 8-9: As mentioned before, every interface test case should check
whether the object implements the tested interface. It is the easiest test
you will ever write, and it is one of the most important ones.
- Line 11-15: This test is equivalent to the test we wrote before, except that
we do not create the sample instance by using the class, but using some
indirection by asking the makeTestObject() to create one for us.
- Line 17-22: In interface tests it does not make much sense to test the
accessor and mutator method of a particular attribute seperately, since
you do not know how the data is stored anyway. So, similar to the test
before, we test some combinations of calling the description getter and
setter.
Now that we have an interface and some tests for it, we are ready to create an
implementation. In fact, we will create two, so that you can see the independence of
the interface tests to specific implementations.
The first implementation is equivalent to the one we used in the unit test chapter,
except that we call it Sample1 now and that we tell it that it implements
ISample.
1 class Sample1(object):
2 """A trivial ISample implementation."""
3
4 implements(ISample)
5
6 # See ISample
7 title = None
8
9 def __init__(self):
10 """Create objects."""
11 self._description = ''
12
13 def setDescription(self, value):
14 """See ISample"""
15 assert isinstance(value, (str, unicode))
16 self._description = value
17
18 def getDescription(self):
19 """See ISample"""
20 return self._description
The second implementation uses Python’s property feature to implement its
title attribute and uses a different attribute name, __desc, to store the data of the
description.
1 class Sample2(object):
2 """A trivial ISample implementation."""
3
4 implements(ISample)
5
6 def __init__(self):
7 """Create objects."""
8 self.__desc = ''
9 self.__title = None
10
11 def getTitle(self):
12 return self.__title
13
14 def setTitle(self, value):
15 self.__title = value
16
17 def setDescription(self, value):
18 """See ISample"""
19 assert isinstance(value, (str, unicode))
20 self.__desc = value
21
22 def getDescription(self):
23 """See ISample"""
24 return self.__desc
25
26 description = property(getDescription, setDescription)
27
28 # See ISample
29 title = property(getTitle, setTitle)
- Line 26: While this implementation chooses to provide a convenience
property to setDescription and getDescription, it is not part of the
interface and should not be tested in the interface tests. However, the
specific implementation tests should cover this feature.
These two implementations are different enough that the interface tests should
fail, if we would have included implementation-specific testing code. The tests can
now be implemented quickly:
1 class TestSample1(TestISample):
2
3 def makeTestObject(self):
4 return Sample1()
5
6 # Sample1-specific tests are here
7
8
9 class TestSample2(TestISample):
10
11 def makeTestObject(self):
12 return Sample2()
13
14 # Sample2-specific tests are here
15
16
17 def test_suite():
18 return unittest.TestSuite((
19 unittest.makeSuite(TestSample1),
20 unittest.makeSuite(TestSample2)
21 ))
22
23 if __name__ == '__main__':
24 unittest.main(defaultTest='test_suite')
- Line 1-6 & 9-14: The realization of the TestISample tests is easily done
by implementing the makeTestObject() method. We did not write any
implementation-specific test in order to keep the code snippets small and
concise.
- Line 12-19: This is just the usual test environment boiler plate.
To run the tests, you need to make sure to have <ZOPE3>/src in your
PYTHONPATH, since this code depends on zope.interface. Then you can simply
execute the code using
1 python test_sampleiface.py
from the directory containing test_sampleiface.py.
1 Configuration file found.
2 Running UNIT tests at level 1
3 Running UNIT tests from /opt/zope/Zope3/Zope3-Cookbook
4 6/6 (100.0%): test_verifyInterfaceImplementation (...leiface.TestSample2)
5 ----------------------------------------------------------------------
6 Ran 6 tests in 0.004s
7
8 OK
As you can see, you just wrote three tests, but for the two implementations six
tests run. Interface tests are a great way to add additional tests (that multiply
quickly) and they are a great motivation to keep on writing tests, a task that can be
annoying sometimes.
Exercises
- Develop implementation-specific tests for each ISample implementation.
- Change the ISample interface and tests in a way that it
uses a description attribute instead of getDescription() and
setDescription(). Note: The requirement that the description value can
only be a regular or unicde string should still be valid.