Some notes on Grok philosophy

Taken from the mailing list. Mostly Martijn Faassen's opinion.

[in response to the idea that Grok-based component separate grok-specific code from reusable code into separate modules]

This is not a Grok-like approach. I think I need to go into this idea some more, as this idea sounds so simple and straightforward to apply, but it really isn't.

One important idea of Grok is that development agility is badly hurt by this separation. Instead of having to be in one place, you need to be in two places (implementation and configuration). That the second place is a separate Python file or a ZCML file doesn't really matter very much - I believe the fact that this second place exists in the first place that's hurting agility the most, not the particular spelling used.

In most cases Grok actually deduces the need for registration from the base class used (for a view, adapter, etc), together with some class annotations (which are optional as they have defaults). Most of the time in Grok-based code, configuration information cannot be separated from the code that is being configured. Code and configuration are actually the same thing, and that's an essential characteristic of Grok's approach.

(note that configuration does happen at a very different stage than code import or execution time - the code is introspected and registrations are made during this introspection phase. This actually happens during the same phase as ZCML registrations occur, as it's triggered from ZCML. I.e,. this is a good idea of ZCML we don't want to give up)

Now if the need to reuse implementation and not the accompanying configuration were very common, the effort to separate configuration from code makes sense. I do not consider this to be common, however: it's the exceptional case where you want to reuse a package and not reuse its configuration. ZCML's approach to enable such separate reuse, separation of configuration information from the code, I consider to have a high cost in agility. That is why I think we need to come up with alternative solutions, instead of separation to tackle the issue of overriding or skipping configuration.

ZCML has some other important beneficial properties besides the potential it has for reuse of code without its configuration. As I already mentioned above, a concept Grok carries over is making configuration happen in a separate phase. Moreover, ZCML is a limited, explicit language. It makes the act of configuration very clear. This really forces developers to think about the act of configuration, and about what's configuration and what's not (and frequently disagree, but at least think about it). This has been extremely helpful.

Doing configuration with Python code runs the risk a thousand inconsistent API will grow and configuration code will become an unpredictable mess. I think our approach with Grok has shown a way to avoid those risks. Grok, for the most part, isn't expressing ZCML configuration directives in the equivalent Python. It's about weaving the configuration information declaratively together with the actual code that it is about.

Choices to separate or integrate happen frequently in programming language design. Docstrings or external API documentation? Separate header declarations versus deducing this from the class declaration? Doctests in docstrings or in separate files? For RMI, interface description languages or introspection?

Related to this is explicit versus implicit - explicit static type or implicit dynamic types? Explicit interfaces or informal protocols?

There is no simple answer about what the right way to go is in any of these cases. It depends on the circumstances. Generally the choice to make something more explicit and more separate has a cost in agility, with hopefully a gain in explicitness and reconfigurability. The choice to integrate something tends to mean a gain in agility and to reduce conceptual load on the developer. There are a lot of trade-offs.

Grok is in the happy position of being able to explore integrated configuration information with an explicit configuration already in place to learn from. We also let go of some of the benefits of ZCML at the same time, hopefully to regain them, later, in a different way.

It also means that the ZCML way of thinking about configuration doesn't really work for Grok - applying the idea of separating the code that is being configured from the configuration information sounds like a relatively straightforward simple approach from the perspective of someone used to ZCML. It would also, in my opinion, remake Grok into something completely different.