ZCML Filtering Support

Author

Problem

Currently, the Zope configuration system allows configurations to be overridden, but not disabled. The only way to disable configuration is to avoid loading it.

Eggs, the emerging Python packaging system, is of increasing importance for Python and Zope development. It is likely that Zope development and distributions will leverage eggs. One of the things that make eggs attractive is their ability to represent dependencies. Installation systems like easy install and zc.buildout can automatically download and install dependencies, making package installation much easier than with lower-level installation mechanisms. When installing Zope packages, it will be increasingly attractive to load configuration for dependencies semi-automatically. I don't think this will be done by the configuration system directly, but rather by individual package configurations. For example, if package X depends on package Y, I expect it to load package Y's configuration as part of it's own configuration. This is already a common practice even without eggs. As more and more configurations are loaded this way, it will become harder to avoid loading configurations.

There are some kinds of configuration that can't be overridden. The overriding mechanism is based on conflicts. Some configurations, most notably subscribers, never conflict and can't be overridden.

Proposal

I propose to provide a way to omit configuration actions based on action filtering. Each action will have a set of "keywords". It will be possible to define keywords explicitly. Grouping contexts will be able to define a keyword that is added to any actions defined within the context, For example, in ZCML, the configure tag will get a keyword attribute that allows a keyword to be defined:

      <configure keyword="zope.app.dublincore subscribers">
         ...
      </configuration>

To add additional keywords, simply nest configuration:

      <configure keyword="zope.app.dublincore subscribers">
         <configure keyword="foo">

         ...
         </configuration>
      </configuration>

The keywords will not apply to actions resulting from the include directive.

In addition, some keywords will be generated automatically. For example:

If a configuration is loaded from a package via an include, then the actions computed from the configuration will get the keyword consisting of the token "PACKAGE:", the package name, and the file name joined by tabs.
Actions with discriminators consisting of strings, will be have a keyword consisting of the token "DISCRIMINATOR:" followed the discriminator values, joined by tabs. (We will convert all discriminators to use strings.)
Actions with None descriminators will get the keyword "NONCONFLICTING".
Certain types of actions will have other automatic keywords. For example, handler actions will get keywords consisting of "HANDLER: followed by the interface identifiers, joined by tabs.

Facilities for including configurations, such as the zcml include directive will provide a way to filter actions. For example, the ZCML include directive might be changed to support subdirectives that provide filtering based on regular expressions, exact matches, or calling python objects:

     <include package="zope.app">
       <exclude>zope.app.dublincore subscribers</exclude>
       <exclude python="somepackage.exclusions" />
       <exclude regex="^__package__\tfoo\.*" />
     </include>

When the include is performed, the resulting actions are filtered based on the exclude directives.

We will also provide an analysis tool that provides for analysis of configuration databases. It would be made available as a Python library that allows a set of actions to be loaded, by performing the first configuration phase. It would yield an action sequence that could be displayed and queried. (We may already have this and may just need to make it available in a more accessible way.):

      >>> import zope.configuration.xml
      >>> actions = zope.configurat.xml.actions('site.zcml')
      >>> for action in [action for action in actions
      ...                if '__nonconflicting__' in action.keywords]:
      ...    print action

This would allow inspection and querying of configurations without actually performing any of the configuration actions.

Alternative Proposal

It has been proposed in the past in ZCMLEnhancements that XPATH be used to filter configuration. I think this has some problems:

It locks us into am XML-based configuration syntax. Using an XML syntax has been fairly unpopular. I'm convinced that we need to allow people to implement configuration in Python and I've proposed as much in the ZopeConfigurationEggSupport. XPath? obviously won't work for filtering Python-based configuration.
It requires XPath? support in the configuration system. This adds a significant dependency. We certainly don't include XPath? support now and including it will likely add a dependency on libxml/libxslt, which is still not well supported on some platforms.

comments:

... --TresSeaver?, Mon, 23 Oct 2006 05:29:20 -0700 reply
A couple of thoughts:

Just to flog the expired equine again (and because you keep bringing it up:): I am strongly opposed to having the configuration language be "Turing complete"; I don't want to allow people to meta-program it. I think that our early experience with over-clever unit testing is even more relevant here: making the configuration itself "imperative" will tempt people to cleverness, which is a Bad Thing (tm) in testing, and a Really Bad Thing for application configuration.
Elementtree suppors a subset of XPath? in its find() and findall() node APIs?; I strongly doubt that we want to support usage beyond what ET already allows.

... --Martijn Faassen, Mon, 23 Oct 2006 05:43:45 -0700 reply
ElementTree? claims to support a subset of XPath? in its find() and findall() APIs?, but that's not actually true - it's not a real XPath? subset. Instead, it supports something XPath?-like, which may be fine for the purposes discussed.

... --TresSeaver?, Mon, 23 Oct 2006 05:52:25 -0700 reply
Yet another one:

Allowing only "exclusions" to be spelled within the '
' directive makes the configuration fragile against upstream changes: it would be more robust to have (at least as an option) the ability to "whitelist" the keywords wanted (the proposed exclude subdirective is a "blacklist"). Perhaps:
```
     <include package="some.package">
       <only>zope.app.dublincore subscribers</only>
       <only python="somepackage.inclusions" />
       <only regex="^__package__\tfoo\.*" />
     </include>
```