Package Dependencies Simplification

When we started converting the Zope 3 tree to eggs, we noticed many complicated interdependencies among the packages, some of which are unnecessary. However, no tool or graph has been created showing the dependencies of a given package.

The first step in this task would be to create a tool that can inspect any package for its complete dependency tree and creates a graph for it. This graph could be in plain text or be visualized using tools like graphviz.

More notes: PackageDependenciesTool

During the second step, all dependencies would be evaluated. If unnecessary dependencies are detected, they should be documented and a short proposal be written explaining how to resolve the dependency.

This work might also include splitting configuration files to support different levels of functionality. For example, when writing an application, you often want the API of a particular package, but not the browser views. But because the main "configure.zcml" file includes the browser one, you have no choice. It would be good to split browser registrations from the other registrations completely. The browser code often also adds dependencies, which can then be split up as well.

• Stephan Richter (can mentor)
• Marius Gedminas
• Graham
• Rocky

Package Listing

Create a listing of all packages in PyPI? and the Zope repository. This is a documentation task. The goal is to create a Web page or set of pages that provides beginners with a quick overview what the various packages are about.

• Nate Aune
• Michael Bernstein

Result of comprehensive PyPI? search of all Zope and Zope3 related packages (338 packages): ZopeInPyPI

48 packages that lack extremely basic info (missing (ie. equals None, an empty string, or UNKNOWN) one of the following: summary, author, author email, or homepage): BadPackages

Security Introspection

Zope 3 comes with a very good security mechanism. However, security has eroded over the years, because components were registered publicly more freely than desired. For example, on many Zope 3 sites you can freely browse to ++skin++Rotterdam.

With a system as complex as Zope 3, it is very hard to find all registrations and evaluate them for every possible context.

To solve this problem, a tool needs to be developed, that provides various means (read pages) to evaluate the available components and their security settings.

• Stephan Richter (can mentor)
• Adam
• Markus

"Real" Test Browser Implementation

The Zope test browser is a wonderful tool that allows us to test complete Web applications using our usual testing environemnt, i.e. doctests. Unfortunately, the underlying framework (mechanize) does not support Javascript, so that modern Web sites cannot be tested.

Benji started a new implementation of the test browser API that controls a real Firefox browser. He uses the MozLab? extension for Firefox, which starts a telnet server in Firfox to which someone can connect. Once connected, the entire browser can be controlled via Javascript calls.

The task during the sprint will be to complete this work.

Having the testbrowser API drive a real browser has other great benefits. For example, doctests can be used to generate real end user documentation including screenshots. Another possibility is to create auto-generated screencasts.

• Stephan Richter (can mentor)
• Justas
• Rocky

Static APIDOC

Many years ago -- actually during the NeckarSprint (2005) -- we strted development on a static version of APIDOC. The idea is that you can create a file-based copy of APIDOC that can be used anywhere.

Many people have expressed a desire to have a static version of the APIDOC. The tool is about 95% done and only a few problems have to be solved. You will become a Zope community hero if you get it working and uploaded to a Web site! :-)

• Stephan Richter (can mentor)

ZCML Optimizations

A significant problem in Zope 3 development is the server startup time. On a full Zope 3 system, the server starts in about 4 seconds and in a minimal egg-based application in about 1.5 seconds on my machine. Of course, those times are minuscule compared to the start-up time of Java-based application servers such as BEA Weblogic which takes up to 30 minutes to start. But Java can do package reloads on the fly, because Java packages do not have global variables that can be manipulated.

While this problem is somewhat mitigated by using test-driven development and the fact that resources and templates are reloaded on the fly, it is still an annoyance when developing. So the question is: How much can we improve the start-up time.

Some code profiling has determined that most of the time (50%) in the start-up process is lost in parsing, converting and validating ZCML directives with their schemas. Thus, this startup problem is not purely a Zope 3 problem, but one that affects everyone using ZCML.

The basic approach is to reduce the ZCML processing time. This can be accomplished by storing some binary representation of the ZCML, similarly to *.pyc files in Python. The solution would be to store a pickle of each parsed directive, namely the action and its arguments. There would be one pickle file fore each ZCML file. When the ZCML file changed, the pickle would be updated. Pickle loading would be much faster than pure ZCML loading, since no XML-parsing, value conversion and schema validation would be necessary.

I have recently investigated this approach. In general I think it is very feasible. The biggest problem is that not all converted values are picklable, such as module references. This special case can be fairly easily handled with some special logic. Overall I think that we can reach a speedup of about 50%, which would mean 2 seconds on a full Zope 3 installation and 0.8 seconds on a simple egg-based application.

• Stephan Richter (can mentor)
• Marius Gedminas

A prototype is available at http://mg.pov.lt/blog/foliage-sprint-day-2.html

Variuos Z3C Form Tasks

This might not be considered a pure Zope 3 topic, since formlib is still the dominant form framework for Zope 3, but z3c.form is quickly evolving in superceding formlib.

While z3c.form is pretty feature complete compared to zope.formlib and zope.app.form, several add-on extensions had been built over the last months that make formlib very comfortable to use. So here are some tasks that could be completed to make z3c.form more powerful:

• Internationalization

  While I originally marked pretty much all human-readable strings as message ids, it turns out that the message ids are not translated. Also, some strings in z3c.formui are not properly marked. It would be also nice to internationalize the form demos and localize them to a couple languages as we have enough foreign speakers around during the sprint.

• Composite Widgets

  It is a common use case that simple text lines want to be entered using more structured methods. For example, a US phone number which always has the number combination xxx-xxx-xxxx or a US social security number that has the form xxx-xx-xxxx.

  A composite widget is a widget that uses a form to collect the various parts of the data field and then combines it to the raw format of that field.

• Session-based Widgets

  It is sometimes necessary to store a value for a field temporarely in a session. The most obvious use case is uploading an image. Let's say you have a widget allows uploading a new image, but also wants to show that image (or a thumbnail) right away in the form. The image cannot be stored on the content, since the form has not yet been submitted, but must be available publically, so the image can be retrieved for display.

• Object Widget

  The object widget caused a lot of grief in the previous form libraries. For z3c.form we decided not to support the object widget an promoted sub-forms instead. But by combining composite and session widgets and providing additional hooks in the form of adapter lookups, I think a very functional object widget could be developed.

• Widgets for Sequences/Dictionaries having Simple Value Types

  One feature that z3c.form currently does not provide is a widget for a sequence with value type TextLine?. It would be really nice to have this. In fact, I think with some small generalizations, we could easily provide widgets for sequences of all sorts of value types.

• Stephan Richter (can mentor)

• Randy Crafton

• Justas

• Martin

Modular Zope 3 Management UI

Several Zope 3 developers are unsatisfied with the current state of the management UI, commonly known as Rotterdam or ZMI for Zope 3. This is mainly due to the fact that the current ZMI does not fulfill today's use cases.

While talking with Roger, who will be leading this effort, we identified two major use cases:

1. The new ZMI needs to allow the configuration and management of the application.
2. The new ZMI needs to be pluggable, so that it can be easily adapted to provide a customized version of itself.

Both of those use cases are also use cases of the current ZMI, but the current ZMI's scope is even larger, for example supporting Web-based development. We also think that the current ZMI does not solve the second use case very well.

This task can be split up into several chunks to ensure maximal reusability.

The first task is to develop a generic base skin that supports our new UI concepts, including pagelets, viewlets and z3c.form. This skin can be used by all developers to get quickly started with Zope 3 skin development.

The second task is to build a pluggable UI that can be used as a base for many different applications. The application that we will develop first will be a simple management UI.

• Roger Ineichen (can mentor)
• Randy Crafton
• Michael Bernstein
• Roland

Atom Publishing Protocol

Atom (IETF RFC 4287) and the Atom Publishing Protocol (in the IETF RFC Editor's queue, which is the final step) are of growing importance for the interoperability of various sorts of content systems, with many diverse implementations already deployed such as Wordpress, Blogger, and Google's GData? APIs?. It has become the canonical REST protocol. Test suites exist in Python, as do other Python implementations, including one for Django. A general-purpose Zope3 implementation will make deploying RESTful? APIs? much easier.

• Michael Bernstein

Example Web Application with Minimal Dependencies

How simply can we (for example) make a clone of http://uselessaccount.com (you can only create an account, login/logout, or edit your account)? What dependencies can we eliminate and still have a working web application? This could be a complementary project to both the Package Dependency Simplification and Modular Zope 3 Management UI.

• Michael Bernstein
• Randy Crafton

Query Language and Query Engine for Zope3

Integrate Z3 with Routes to make Zope more suitable for RDB applications

• Graham Stratton
• Michael Bernstein

Make progress with JQuery? and JSON integration

• Roger Ineichen

Add Your Topic Here

• First Participant
• Second Participant