Site Themes

Zope Site Themes

Status

IsProposal

Author

Paul Everitt

Summary

Re-skinning a site is one of the first activities in customizing a deployment. Customers want their own "corporate identity". Currently, "skins" cover two divergent goals: corporate ID and dynamic generation of markup from content.

This proposal splits off the former into an idea called a site theme. This is a standard corporate identity look-and-feel, applied to every page in a site. The proposed implementation relies on very simple ideas and standard machinery to avoid even touching the files that constitute a theme.

Specifically, a rule file controls how dynamic content gets merged into boxes in the theme file using normal HTML node identifiers. The files created by the Dreamweaver person are untouched and contain no template markup.

The Problem

CMF and Zope 3 have a concept of skinning, where different template sets can be created, installed, and used in a site. A skin acts both as the place where component developers render dynamic content and designers change look-and-feel.

In general, corporate id isn't tied deeply to the CMS. Hundreds of staff hours have gone into getting it right and it is probably already in use in non-Zope systems. There is a bunch of static markup and boxes on the screen where dynamic stuff will go.

For the dynamic stuff, the customer might want to customize the way it looks beyond what CSS can do. But usually they don't, and even if they do, they refuse to learn yet-another-framework to accomplish this. Instead, they just tell the consultant to do it. Thus, the markup on a navigation tree isn't in scope for the corporate id person, who usually appears on a project for approximately a single day and dictates the guidelines in the company style book.

The current skin approach has some drawbacks:

1) You got your chocolate in my peanut butter. The main_template.pt file is the entry point for how a basic corporate id is assembled for all pages. One look at the first 30 lines of this file, in CMF (and possibly Zope 3) would frighten most Dreamweaver people. It would be better if the responsibilities of corporate id and dynamic generation were completely separated.

2) Too busy. Most designers don't want to learn another template system and another API for grabbing content. They don't really even want to see such programming droppings when they open the "their" HTML file (ZPT).

3) Upgrades. A new version of Zope/CMF ships. The main_template has some new capabilities or bugs fixed in the previous one. However, since the first thing a deployment does is corporate id, there is a diff/merge that has to take place.

4) Opt-in. All templates in all core code and all add-on products have to decipher the under-documented, un-enforced system of macros and slots in order to leverage the corporate id work in main_template. If you want to apply a consistent look and feel on all pages, you should be able to impose it, whether the view template agreed or not.

5) Performance. Modern customer sites have tons of images, CSS files, JavaScript? files to constitute the corporate id on every page. There should be an option to allow systems outside Zope (Apache, Twisted in Zope 3.2) to serve all of this without mastering the art of cache tuning.

6) Limits to CSS. CSS can do a lot of corporate id, but there's a lot it can't do. You can't insert a copyright statement with CSS, for example. Even if you could, most designers don't think that way and they would thus have to re-do the implementation of their existing corporate id artifacts.

Goals

The solution chosen should be evaluated against some set of criteria that are independent of the implementation. This proposal was written to meet the following goals:

1) Don't make corporate ID people learn anything new. They only needed to understand the same ideas CSS uses (ways to identify blocks via id attributes).

2) Focus gives simplicity. If we can focus on a single audience and the plausible ways that audience works, we can eliminate solutions that, while they fit our brain, are alien to the audience. This proposal is aimed at the Dreamweaver person that controls the organization's existing corporate identity.

3) Universal. The corporate id is not optional. It gets imposed on every page in a site.

4) Improve the consulting process. Imagine a consultant walks in, goes to the current website, and does a "Save as..." in the browser. An HTML file and a bunch of linked stuff gets saved. They edit a small (5 line) rule file, and reload the CMS page, and the corporate id appears.

That's the start. Then, during the lifetime of the process (software changes and upgrades, wild swings as APIs? move around, etc.), the results of "Save as..." never change. One day the Dreamweaver person shows up with a new pile of corporate id. It gets saved on the server, the rule file doesn't change, and instantly the site is re-themed.

5) Off-the shelf ideas, off-the-shelf machinery. The world has enough template syntaxes, architectures, and implementations. If possible, leverage a commodity idea instead of "innovating", which will require training the Dreamweaver person and maintaining a parser.

6) Useful outside Zope. Some might want to improve performance or leverage other platforms. For example, Apache or Twisted might be used to serve the HTML, CSS, JS, and images.

7) Very normal to debug. Site themes are a simple activity. If this simple activity gets merged in with not-simple activities, it becomes more challenging to isolate problems. The site theming solution should, to the degree possible, be easy to isolate, test, and debug.

8) Performance. By removing content rendering from site theming, the latter can focus on simple, well-known technologies with well-understood, predictable performance characteristics and tunability.

Proposal

This proposal recommends a box-filling approach based on HTML id attributes. A site theme is an HTML file with its accompanying images, CSS, and JS. The site theme has block elements, or boxes, that get replaced with block elements coming from the app server.

This merging is controlled by a rule file, written either as a Python dictionary or an XML file. (ElementTree? or lxml could allow the same code to walk Python/XML data.) For example:

          rules = {
            'breadcrumbs':'breadcrumbs',
            'nav':'navigation',
            'body':'contentbody'
          }

Re-theming is simple. Save an existing web page to disk, make a map, and let Apache deliver (no ZCML). The corporate ID is untouched. The map is managed by the integrator, who knows what structures are coming from the app server (or, can mock them up and have them implemented later).

The merging can be done in many ways. If a full XML engine is present, such as lxml/libxml2, the merging can be done with a small XSLT transform. If only ElementTree? or cElementTree are present, the merging can be done using its optimizations for finding a node with an id. Perhaps the fastest approach would be SAX (as this is a simple problem set.)

This merge operation can be done either in mod_python, as a filter on certain MIME types, in Zope 3.2 at the Twisted level, or as part of the Zope runtime itself. For the latter, an event for publishing, as is present in Zope 3.1, helps significantly. This could also be done in mod_perl, mod_php, or perhaps even in a generic transformation module (like mod_transform).

This proposal, though, needs reasonably-parseable site themes (the Dreamweaver part) and content (the generated markup from the app server). The Python extension module for HTML Tidy can help. When lxml wires in the HTML parser, that could also be used.

With this proposal, all "style" stuff will be moved out of the current skin artifacts. This allows the component developers to see less noise (from their perspective).

This proposal could be combined with other proposals as part of a pipeline. For example, an earlier pipeline stage could be responsible for producing well-formed, valid XHTML 1 or 2. Another stage could be responsible for generating the markup for the navigation tree or site menu.

Scope

1) Generic, but passively. This approach and specification can be implemented many ways. It could be part of some WSGI common library, for example. It could work also in PHP, for example, using existing machinery. However, this project doesn't plan to evangelize broader use, unless there is legitimate interest.

2) Not dynamic. This is not a system to generate markup from Zope objects. It could be used in conjunction with such a system. However, everything about this can be evaluated without pipelines.

Risks

1) Scope creep. Perhaps the site theme should generate navtrees? Perhaps it should do complex page layouts, like the wonderful work in CPSSkins?? Perhaps the rule language should allow conditional determination? These ideas shouldn't be approached within the scope of site themes but should be handled at an earlier stage in processing.

2) XML machinery. Which parser? Use SAX? Don't use XML but regex? All of these are useful questions but not central to the proposal. Stated differently, this shouldn't be rejected because it might (or might not) have an XML perspective on markup processing.

3) Both means neither. This can work in Apache and in Zope. Or even Twisted in Zope 3.1. The danger is that, by trying to support both, neither are done particularly well and thus neither get broad acceptance.

4) Overlap with other efforts. Most notably, CPSSkins? has a concept of themes. Evaluation of this site themes proposal should neither restrict nor be restricted by evaluation of Jean-Marc's tremendous work.

5) Requires well-formed markup. Under nearly any circumstance (unless regex is used), a site theme will need to be reasonably clean (X)HTML. So will the content, which can be a harder problem (most Zope documents/pages don't validate markup on input). Tidy and libxml2 probably surpass our threshold for tolerance, but will never reach 100%.

Open Questions

1) What if some pages in a site have one corp id and some have others? We can call YAGNI or have some kind of URL mapping, like Cocoon. We could always have Apache config handle this case.

2) Link and style elements from the CMS can be necessary in the output.

3) We could even eliminate the need for id attributes in the theme if we allowed a full XPath? syntax. This, though, would eliminate minidom, ElementTree?/cElementTree, and possibly SAX as implementation choices. The merge system would require a real XPath? engine, basically.

Sample Implementation

I have an implementation for mod_python that uses lxml. It requires around 20 lines of Python and 30 lines for the XSLT. The XSLT is generic; it reads a rule file that differs for each theme, but the XSLT is never seen by the integrator.

I can do another implementation using just ElementTree?/cElementTree. I'm not very good at SAX, but I could try making an implementation there.

Tres Seaver did a pipeline implementation at EuroPython? 2005. This could be used for a non-Apache evaluation.

I didn't make a stab at removing all "corporate id" from the default Zope 3 skin. This would be an interesting exercise, to see how much complexity decreases and, perhaps, performance increases by this.