Chapter 26
New Principal-Source Plug-Ins

Difficulty

Sprinter

Skills

You should have a basic understanding of the Zope 3 component architecture.
It is necessary to understand the purpose and differences between permissions, roles and principals.
Basic knowledge about the Authentication Service. Optional.

Problem/Task

Many systems provide their own mechanisms for authentication. Examples include /etc/passwd, LDAP, NIS, Radius and relational databases. For a generic platform like Zope it is critically necessary to provide facilities to connect to these external authentication sources.

Zope 3 provides an advanced Authentication Service that provides an interface to integrate any external authentication source by simply developing a small plug-in, called a principal source. In this chapter we create such a plug-in and register it with the Authentication Service .

Solution

While one can become very fancy in implementing a feature-rich principal source implementation, we are concentrating on the most simple case here. The exercises point out many of the improvements that can be done during later development.

The goal of this chapter is to create a file-based principal source, so that it could read a /etc/passwd-like file (it will not actually be able to read passwd files, since we do not know whether everyone has the crypt module installed on her/his machine). The format of the file that we want to be able to read and parse is (users are separated by a newline character):

1 login:password:title:other_stuff

Let’s now turn to the principal source. A component implementing ILoginPasswordPrincipalSource (which extends IPrincipalSource) promises to provide three simple methods:

getPrincipal(id) - This method gets a particular principal by its id, which is unique for this particular source. If no principal with the supplied id is found, a NotFoundError is raised.
getPrincipals(name) - This method returns a list of principals whose login somehow contains the substring specified in name. If name is an empty string, all principals of this source are returned.
authenticate(login,password) - This method is actually required by the ILoginPasswordPrincipalSource interface and provides authentication for the provided principal. There are other ways to implement authentication for these principals, but they add unnecessary complexity. None is returned, if no match is made.

The next step is to provide an implementation of ILoginPasswordPrincipalSource for password files. Create a new sub-package called passwdauth in the book package. Now the first step is to define the interfaces, as usual.

26.1 Step I: Defining the interface

“What interface do we need?”, you might wonder. In order for a file-based principal source plug-in to provide principals, we need to know the file that contains the data; knowing about this file is certainly part of the API for this plug-in. So we want to create a specific interface that contains a filename. If we make this attribute a schema field, we can even use the interface/schema to create autogenerated add and edit forms.

In the passwdauth directory add an interfaces.py file and add the following contents:

1  from zope.schema import TextLine
2  from zope.app.i18n import ZopeMessageIDFactory as _
3
4  from zope.app.pluggableauth.interfaces import IPrincipalSource
5
6  class IFileBasedPrincipalSource(IPrincipalSource):
7      """Describes file-based principal sources."""
8
9      filename = TextLine(
10          title = _(u'File Name'),
11          description=_(u'File name of the data file.'),
12          default = u'/etc/passwd')

Line 1: Here we have the usual imports of the TextLine field for the filename property.
Line 2: This is the typical I18n boilerplate (not much though); all text strings wrapped by the underscore function will be internationalized, or in other terms localizable.
Line 4: Our file-based principal source is still of type IPrincipalSource, so let’s make it the base interface.
Line 9-12: Typical internationalized text line field declaration, making /etc/passwd the default value (even though the product will not work with this file due to the crypt module issue). You might want to add a different default, also based on the operating system you are on.

26.2 Step II: Writing the tests

The next step is to write some unit tests that assure that the file parser does its job right. But first we need to develop a small data file with which we can test the plug-in with. Create a file called passwd.sample and add the following two principal entries:

1 foo1:bar1:Foo Bar 1
2 foo2:bar2:Foo Bar 2

Now we have a user with login foo1 and one known as foo2, having bar1 and bar2 as passwords, respectively.

In the following test code we will only test the aforementioned three methods of the principal source. The file reading code is not separately checked, since it will be well tested through the other tests.

Create a tests.py file and add the code below.

1  import os
2  import unittest
3
4  from zope.exceptions import NotFoundError
5
6  from book import passwdauth
7
8
9  class PasswdPrincipalSourceTest(unittest.TestCase):
10
11      def setUp(self):
12          dir = os.path.dirname(passwdauth.__file__)
13          self.source = passwdauth.PasswdPrincipalSource(
14              os.path.join(dir, 'passwd.sample'))
15
16      def test_getPrincipal(self):
17          self.assertEqual(self.source.getPrincipal('\t\tfoo1').password, 'bar1')
18          self.assertEqual(self.source.getPrincipal('\t\tfoo2').password, 'bar2')
19          self.assertRaises(NotFoundError, self.source.getPrincipal, '\t\tfoo')
20
21      def test_getPrincipals(self):
22          self.assertEqual(len(self.source.getPrincipals('foo')), 2)
23          self.assertEqual(len(self.source.getPrincipals('')), 2)
24          self.assertEqual(len(self.source.getPrincipals('2')), 1)
25
26      def test_authenticate(self):
27          self.assertEqual(self.source.authenticate('foo1', 'bar1')._id, 'foo1')
28          self.assertEqual(self.source.authenticate('foo1', 'bar'), None)
29          self.assertEqual(self.source.authenticate('foo', 'bar1'), None)
30
31  def test_suite():
32      return unittest.makeSuite(PasswdPrincipalSourceTest)
33
34  if __name__=='__main__':
35      unittest.main(defaultTest='test_suite')

Line 1, 12-14: The reason we imported os was to be able to get to the directory of the code as seen in line 12. Once we have the directory it is easy to build up the data file path and initialize the principal source (line 13-14).
Line 16-19: Test the getPrincipal(id) method. The last test checks that the correct error is thrown in case of a failure. The full principal id is usually a tab-separated string of an earmark, the principal source name and the principal id. Since we do not have an earmark or a principal source name specified in a unit tests, these two values are empty and the full principal id has two tab characters at the beginning.
Line 21-24: The test for getPrincipals(name) mainly tests that the resulting user list is correctly filtered based on the name parameter value.
Line 26-29: The authentication test concentrates on checking that really only a valid login name and password pair receives a positive authentication by returning the principal object.
Line 31-35: This is the usual test boiler plate.

You can later run the tests either using Zope’s test.py test runner or by executing the script directly; the latter method requires the Python path to be set correctly to ZOPE3/src.

26.3 Step III: Implementing the plug-in

The implementation of the plug-in should be straightforward and bear no surprises. The tests already express all the necessary semantics. We only have not discussed the data structure of the principal itself yet. Here we can reuse the SimplePrincipal, which is a basic IUser implementation that contains all the data fields ( IUserSchemafied) relevant to a principal: id, login (username), password, title and description.

Note that in Zope 3 the principal knows absolutely nothing about its roles, permissions or anything else about security. This information is handled by other components of the system and is subject to policy settings. Now we are ready to realize the principal source. In the __init__.py file of the passwdauth package we add the following implementation:

1  import os
2  from persistent import Persistent
3
4  from zope.exceptions import NotFoundError
5  from zope.interface import implements
6
7  from zope.app.container.contained import Contained
8  from zope.app.location import locate
9  from zope.app.pluggableauth import SimplePrincipal
10  from zope.app.pluggableauth.interfaces import IContainedPrincipalSource
11  from zope.app.pluggableauth.interfaces import ILoginPasswordPrincipalSource
12
13  from interfaces import IFileBasedPrincipalSource
14
15  class PasswdPrincipalSource(Contained, Persistent):
16      """A Principal Source for /etc/passwd-like files."""
17
18      implements(ILoginPasswordPrincipalSource, IFileBasedPrincipalSource,
19                 IContainedPrincipalSource)
20
21      def __init__(self, filename=''):
22          self.filename = filename
23
24      def readPrincipals(self):
25          if not os.path.exists(self.filename):
26              return []
27          file = open(self.filename, 'r')
28          principals = []
29          for line in file.readlines():
30              if line.strip() != '':
31                  user_info = line.strip().split(':', 3)
32                  p = SimplePrincipal(*user_info)
33                  locate(p, self, p._id)
34                  p._id = p.login
35                  principals.append(p)
36          return principals
37
38      def getPrincipal(self, id):
39          """See `IPrincipalSource`."""
40          earmark, source_name, id = id.split('\t')
41          for p in self.readPrincipals():
42              if p._id == id:
43                  return p
44          raise NotFoundError, id
45
46      def getPrincipals(self, name):
47          """See `IPrincipalSource`."""
48          return filter(lambda p: p.login.find(name) != -1,
49                        self.readPrincipals())
50
51      def authenticate(self, login, password):
52          """See `ILoginPasswordPrincipalSource`. """
53          for user in self.readPrincipals():
54              if user.login == login and user.password == password:
55                  return user

Line 2 & 14: Make sure the principal source object itself is persistent, so that it can be stored in the Authentication Service.
Line 4: The NotFoundError is a Zope-specific exception, so we need to import it.
Line 7 & 14: Since the principal source is stored inside an authentication service, we need to make it an IContained object.
Line 8: The locate() method helps us assigning a location to objects, in this case principals. Since principals are contained by principal sources, we need to assign a parent and a name to them when they are created.
Line 9: Here you can see where the SimplePrincipal is defined. There is really no need to implement our own version, even though it is a persistent class - we never add it to any object in the ZODB anyways.
Line 10-13, & 18-19: Import the three principal source interfaces we promise to implement in our new principal source. The IContainerPrincipalSource makes sure that the principal source can only be added to a pluggable authentication service and nowhere else.
Line 21-22: We need to make sure the filename attribute always exists; optionally it can even be passed to the constructor; we will make use of this fact in the autogenerated add form.
Line 24-36: The readPrincipals() method does all the heavy lifting as it is responsible for reading and “parsing” the file. It contains all the logic for interpreting the file format. readPrincipals() is just a helper method and is therefore not defined in any interface.
- Line 25-26: In the first if statement the algorithm checks that the file really exists and return an empty list if it does not. This prohibits Zope from crashing if the file is not found, which is desirable in case you just made a simple typo and now you cannot access your Zope, because any authentication check will fail, since it passes through this code for every authentication call.
- Line 29: As mentioned before we assume that there is one line per user.
- Line 30: Let’s ignore empty lines, they just cause headaches.
- Line 31-32: Another assumption is made; the entries in the file correspond directly to the arguments of the SimplePrincipal constructor, which is valid as long as the constructor signature of SimplePrincipal does not change.
- Line 33: Assign a location to the principal, so that we know where it came from.
- Line 34: The principal’s login is generally different from its id field. Since we do not just want to support /etc/passwd files, we are not going to reuse the Unix user id, but simply use the login for its id.
Line 38-44: This implementation of the getPrincipal() method reads all principals in and checks whether one with a matching id is found; if not, raise a NotFoundError. This is of course horribly inefficient and one should use caching - see Exercise 5 at the end of this chapter.
The principal id that is passed into this method argument really exists of three parts separated by a tab-character. The first part is the earmark (or unique id) of the authentication service, the second the name of the principal source and the third the id of the principal (line 38). However, we are only interested in the last part, which we use for comparison.
Line 46-49: Again we simply use the readPrincipals() result to built up the list of matching principals.
Line 51-55: The authenticate() method simply wades through all the users and tries to find a matching login/ password pair. When a match is found, the principal object is returned. Note that Python returns None, if no return value is specified, which is the case if no match was determined.

You should now run the unit tests to make sure that the implementation behaves as expected.

26.4 Step IV: Registering the Principal Source and Creating basic Views

We now have to register the PasswdPrincipalSource as content and create a basic add/edit form, since we need to allow the user to specify a data file. Create a configuration file ( configure.zcml) and add the following directives:

1  <configure
2      xmlns="http://namespaces.zope.org/zope"
3      xmlns:browser="http://namespaces.zope.org/browser"
4      i18n_domain="demo_passwdauth">
5
6  <content class=".PasswdPrincipalSource">
7    <factory
8        id="zope.app.principalsources.PasswdPrincipalSource"
9        />
10    <allow interface=".interfaces.IFileBasedPrincipalSource"
11        />
12    <require
13        permission="zope.ManageContent"
14        set_schema=".interfaces.IFileBasedPrincipalSource"
15        />
16  </content>
17
18  <browser:addform
19      schema=".interfaces.IFileBasedPrincipalSource"
20      label="Add file-based Principal Source in /etc/passwd style"
21      content_factory=".PasswdPrincipalSource"
22      arguments="filename"
23      name="AddPasswdPrincipalSourceForm"
24      menu="add_principal_source" title="/etc/passwd Principal Source"
25      permission="zope.ManageContent"
26      />
27  <browser:editform
28      schema=".interfaces.IFileBasedPrincipalSource"
29      label="Edit file-based Principal Source"
30      name="edit.html"
31      menu="zmi_views" title="Edit"
32      permission="zope.ManageContent"
33      />
34  </configure>

Line 6-16: Define the principal source as content, create a factory for it and make the appropriate security declarations for the interfaces. While the factory id (line 8) is usually the same as the Python object path, this is not the case here. However, this poses no problem, since the only requirement is that the id is globally unique.
Line 18-26: Create a simple autogenerated add form. We also specify that the filename is the first and only argument for the constructor.
Line 27-33: Matching to the add form, this is a simple edit form for the file name. Plain and simple is enough in this case.

One last step must be taken before the package can be tested: We need to incorporate the package into the system. Therefore add a file named passwdauth-configure.zcml into the package-includes directory having the following content:

1 <include package="book.passwdauth" />

Now (re)start your Zope server and try the new plug-in.

26.5 Step V: Taking it for a test run

After you have restarted Zope, take a Web browser and access http://localhost:8080/. From the contents screen go to the configuration by clicking on the ManageSite link. You are probably being prompted to login. Make sure you are logging in as a user having the zope.Manager role.

If you do not have an Authentication Service yet, then add this service by clicking on the AddService link. Select AuthenticationService and give it the name auth_service. The service will be automatically registered and activated for you. After this is done, you are left in the Registration view of the authentication service itself. In the Add: box on the left side, you should now see two entries, one of which is /etc/passwdPrincipalSource. Click on the new principal source and enter passwd as the name of the principal source.

In the next screen you are being asked to enter the path of the file. Darn, you might think, I do not have a file yet. But don’t worry, we still have the file we used for the tests, so we can reuse it (and we know it works). So enter the following path, replacing ZOPE3 with the path to your Zope 3 installation:

1 ZOPE3/src/book/passwdauth/passwd.sample

After submitting the form you end up in the Contents view of the Authentication Service again. Unfortunately, we have not added a screen yet, telling us whether the file exists and it successfully found users. I leave this exercise for the reader.

Before we can use the new principals, however, we have to assign roles to them. So go to http://localhost:8080/@@contents.html. In the top right corner you will see a Grant menu option. Click on it. In the next screen click on Grantrolestoprincipals. Now you should be convinced that the new principal source works, since “Foo Bar 1” and “Foo Bar 2” should appear in the principal’s list. Select “Foo Bar 1” and all of the listed roles and submit the form by pressing Filter. In the next screen you simply select Allow for all available roles, which assigns them to this user. Store the changes by clicking Apply.

We are finally ready to test the principal! Open another browser and enter the following URL: http://localhost:8080/@@contents.html. You will be prompted for a login. Enter foo1 as username and bar1 as password and it should show you the expected screen, meaning that the user was authenticated and the role SiteManager was appropriately assigned. You should also see User:FooBar1 somewhere on near the top of the screen.

Exercises

The chapter’s implementation did not concentrate much on providing feedback to the user. It would be nice to have a screen with an overview of all of the login names with their titles and other information. Implement such a screen and add it as the default tab for the principal source.
The current implementation requires the passwords to be plain text, which is of course a huge security risk. Implement a version of the plug-in that can handle plain text, crypt-based and SHA passwords. Implement a setting in the user interface that lets the user select one of the encryption types.
Implement a version of the plug-in that provides a write interface, i.e. you should be able to add, edit and delete principals. It would be best to implement the entire IContainerPrincipalSource interface for this, since you can then make use of existing Container code.
It is very limiting to require a special file format for the principal data. It would be useful to develop a feature that allows the user to specify the file format. Implement this feature and provide an intuitive user interface for this feature. (This is a tough one; feel free to make some assumptions to solve the problem.)
Reading in the user data for every single authentication call is quiet expensive, so it would be helpful to implement some caching features. This can be done in two ways: (1) Use the caching framework to implement a cached version of the source or (2) save the list of principals in a volatile attribute (i.e. _v_principals) and check for every call whether the file had been modified since the last time it was read.

Chapter 26New Principal-Source Plug-Ins