home contents changes options help subscribe edit (external edit)

last edited 1 year ago by dmaurer

Motivation

Volatile attributes have several interesting use cases

  • cache the result of expensive computations
  • store unpicklable resource which is object and connection specific

    The primary example is the database connection in Shared.DC.ZRDB.Connection

  • store acquisition dependent information for use in an acquisition-less context

    The primary example has been the use of _v_skindata in the CMF Skinnable implementation

  • store temporary control information relevant for object processing further down in the hierarchy.

    The primary example is the use of _v_is_cp in Archetypes to differenciate the use of manage_afterAdd for either a copy or a move.

Apart from the first use case, all other use cases require a garanteed lifetime.

The ZODB does not garantee any lifetime for volatile attributes. Fortunately, for the use cases, the current ZODB implementation is quite conservative with objects. Usually, it deactivates objects only at transaction or savepoint boundaries and while the loss at a savepoint boundary occasionally breaks one of the use cases above, it happens rarely enough that volatile variables are still in use for these use cases. Sometimes, however, nasty and apparently non-deterministic errors occur due to unexpected loss of a volatile variable.

There are alternative solutions for all affected use cases; however much more complex than volatile attributes. For example the use of _v_skindata has been replaced in CMF by a module level registry indexed by id and thread identifier. As the data references persistent objects, it must not be used by a different connection. As in the next request the thread might be associated with a different connection, the thread specific data must be flushed at the end of the request which prevents the use of some caching optimizations. Furthermore, the current implementation breaks if several Skinnable objects are used in the same request (we have a use case for this).

Therefore, I propose to implement a limited lifetime garantee for volatile attributes.

Solution Outline

The proposed solution gives applications a means to indicate that an object should only be deactivated implicitely at transaction boundaries. We call such objects sticky as they stick more to the cache than normal objects. Stickyness does not affect explicit deactivation (by the application). As volatile attributes vanish implicitely only through deactivation, this effectively gives the volatile attributes of sticky objects a garanteed lifetime until the next transaction boundary. This is enough for all use cases listed above.

Cache garbage collection is slightly modified to distinguish between normal collection and strong collection. Normal collection does not deactivate sticky objects while strong collection can deactivate sticky objects as well.

As outlined above, strong collection is used at transaction boundaries while normal collection is used everywhere else.

Use Cases

We distinguish two use cases

  • class level stickyness

    The class knows that is uses volatile variables in a way requiring transaction wide lifetime and therefore declares stickyness for all its instances.

    This use case supports e.g. the use in CMF Skinnable and in Zopes database adapters.

  • instance level stickyness

    The application needs to temporarily protect a volatile variable.

    This use case supports e.g. the use of _v_is_cp in Archetypes.

Implementation Outline

Stickyness is declared by giving a new persistent attribute _p_sticky a true value. For efficiency reasons, _p_sticky is maintained at "C" level inside the header of persistent objects. Class level stickyness is taken care of during object construction. A later change of the class stickyness does not affect already created instances. del instance._p_sticky will restore the class level stickyness (if any).

The cache garbage collection method gets a new parameter strong. It will invalidate sticky objects only when strong has a true value.

A ZODB Connection will call the cache garbage collection with strong=True only at transaction boundaries.

Risks

  • Handling class level stickyness correctly might be a bit tricky.
  • The behavior of instance stickyness when the class stickyness changes (the stickyness of existing instances does NOT change) might be surprising.
  • When an application sets an instance level stickyness and later does not reset it, the lifetime of the stickyness declaration extends to at least the transaction boundary as the instance will not be deactivated (automatically) before that time. However, it may live arbitrarily longer. It survives deactivation and dies only when the instance is flushed from the connection cache completely. This might be surprising but almost surely will remain unnoticed.

Deliverables



subject:
  ( 11 subscribers )