Memory usage

A place to throw notes on ZODB and Zope memory consumption -

An informative post from DieterMaurer? on the zope list:

The ZODB aimed at ease of integration and ease of implementing simple

multi-threaded applications (such as Web application servers).

To make the second easier, it removed a major fault class: locking

faults in application code. With the ZODB, you need not to worry

about setting locks to make the modification of persitent objects

safe. Because of that, you will usually not have problems with deadlocks

or data inconsistencies due to forgotten locking.

However, this ease has a price: the quite costly ZODB cache maintained

for each connection in a connection pool. To limit RAM consumption,

the ZODB versions before 3.4 had a limit on the number of connections

in the pool. With ZODB 3.4, the number of connections are no longer

limited. Connections exceeding the pool size become temporary connections

the cache of which is destroyed when the connection is closed.

Thus, you can now have a rather small number of ZODB connections

(to hold RAM consumption in check) and a rather large number of workers

to reduce the risk of deadlocks caused by resource exhaustion.

If you are ready to modify ZServer? a bit, you could even create

workers on demand (usually, they are preallocated).

However, this will cost you much of your control over resource usage...