Turning MessageIDs into rocks

Status:

IsImplementedProposal

Author

PhilippVonWeitershausen

Status quo

MessageIDs are implemented as a subclass of unicode. They are typically instanciated using MessageIDFactory.

For example:

    >>> from zope.i18nmessageid import MessageIDFactory
    >>> _ = MessageIDFactory("futurama")
    >>> robot = _(u"robot-message", u"${name} is a robot.")

The default value as well as the mapping for variable interpolation are introspectable:

    >>> robot
    u'robot-message'
    >>> robot.domain
    'futurama'
    >>> robot.default
    u'${name} is a robot.'
    >>> robot.mapping

These values are also changeable, which is especially useful when providing data with which the variables are to be interpolated:

    >>> robot.domain = "planetexpress"
    >>> robot.default = u"${name} is not a robot."
    >>> robot.mapping = {u'name': u'Bender'}

    >>> robot.domain
    'planetexpress'
    >>> robot.default
    u'${name} is not a robot.'
    >>> robot.mapping
    {u'name': u'Bender'}

    >>> from zope.i18n import translate
    >>> translate(robot)
    u'Bender is not a robot."

Problem

The mutability of message ids contradicts the fact that string and unicode objects are immutable. Except that it additionally carries translation information, a message id should behave like a unicode object anywhere else.

The mutability issue becomes very evident and problematic when dealing with security. Since they are not basic objects (immutable objects like string and unicode instances, often called rocks), they are security proxied. Components using message ids returned by other components have to strip security proxies from the message ids before working with them. This is unnecessary.

Having security declarations for message ids is only a somewhat acceptable solution. It would still not solve the mutability problem. We have been treating message ids like unicode strings all along. If it wasn't for translation, we would use unicode. Therefore, message ids should behave like unicode strings in every possible way. That includes immutability. As a bonus, the security system will treat them like basic objects (rocks), making security declarations unnecessary.

Goals

Avoid security proxying of message ids.
Make semantics of message ids similar to those of string and unicode objects.
Retain full functionality.

Proposed solution

The solution I'm proposing makes message ids immutable, in other words they'll be treated as rocks by the security framework. This means that all of the additional attributes on messageid instances are absolutely static. If one is to be changed, a new instance is created.

Consider the example from above:

    >>> from zope.i18nmessageid import MessageFactory, Message
    >>> _ = MessageFactory("futurama")
    >>> robot = _(u"robot-message", u"${name} is a robot.")

Messages at first seem like they are unicode strings:

    >>> robot
    u'robot-message'
    >>> isinstance(robot, unicode)
    True

The additional domain, default and mapping information is available through attributes:

    >>> robot.default
    u'${name} is a robot.'
    >>> robot.mapping
    >>> robot.domain
    'futurama'

The message's attributes are considered part of the immutable message object. They cannot be changed once the message id is created:

    >>> robot.domain = "planetexpress"
    Traceback (most recent call last):
    ...
    TypeError: readonly attribute

    >>> robot.default = u"${name} is not a robot."
    Traceback (most recent call last):
    ...
    TypeError: readonly attribute

    >>> robot.mapping = {u'name': u'Bender'}
    Traceback (most recent call last):
    ...
    TypeError: readonly attribute

If you need to change their information, you'll have to make a new message id object:

    >>> new_robot = Message(robot, mapping={u'name': u'Bender'})
    >>> new_robot
    u'robot-message'
    >>> new_robot.domain
    'futurama'
    >>> new_robot.default
    u'${name} is a robot.'
    >>> new_robot.mapping
    {u'name': u'Bender'}

Last but not least, messages are reduceable for pickling:

    >>> callable, args = new_robot.__reduce__()
    >>> callable is Message
    True
    >>> args
    (u'robot-message', 'futurama', u'${name} is a robot.', {u'name': u'Bender'})

    >>> fembot = Message(u'fembot')
    >>> callable, args = fembot.__reduce__()
    >>> callable is Message
    True
    >>> args
    (u'fembot', None, None, None)

Message IDs? and backward compatability

The change to immutability is not a simple refactoring that can be coped with backward compatible APIs?--it is a change in semantics. Because immutability is one of those "you either have it or you don't" things (like pregnancy or death), we will not be able to support both in one implementation.

The proposed solution for backward compatability is to support both implementations in parallel, deprecating the mutable one. A separate factory, MessageFactory, instantiates immutable messages, while the deprecated old one continues to work like before.

The roadmap to immutable-only message ids is proposed as follows:

Zope 3.1: Immutable message ids are introduced. Security declarations for mutable message ids are provided to make the stripping of security proxies unnecessary.

Zope 3.2: Mutable message ids are deprecated.

Zope 3.3: Mutable message ids are removed.

Implementation and Status

Because truly immutable objects cannot be implemented in Python, the new messages have to be implemented in C. Jim and I paired on this at the Castle Sprint 2004. This proposal serves as a doctest at zope.i18nmessageid/messages.txt.

Step 1 of the above roadmap has been fulfilled: Zope 3.1 has rolled out with the new message implementation. Step 2 has been fulfilled on the Zope 3 trunk which will become Zope 3.2.

Risks

There exists a notable amount of documentation on message ids, most prominently the two printed books (this is why long-term backward compatability is important).

comments:

Implementation status --mgedmin, 2005/10/12 06:46 EST reply
Zope 3 trunk, revision 39091, has an implementation of this proposal, but without the string interpolation syntax.

You can do things like:

  >>> from zope.i18nmessageid import MessageFactory
  >>> from zope.i18n import translate
  >>> _ = MessageFactory('my_domain')

  >>> msg = _('A $message', mapping={'message': 'greeting!'})
  >>> translate(msg)
  u'A greeting!'

  >>> msg = _('A $message') % {'message': 'greeting!'}
  >>> translate(msg)
  u'A $message'