checkzopemem

An alternative to AutoLance & other MonitorScript?'s, tested with zope 2.7 on debian GNU/linux. This is currently keeping this site's zope in check.:

 #!/bin/bash
 # checkzopemem - log zope memory usage and restart when it gets too large
 # run this every minute or so, eg with an entry in /etc/crontab like this:
 # * *     * * *   root    checkzopemem
 # warning: may fail to work if memory blows up too quickly

 MAXSIZE=600000
 INSTANCE=/zope1
 LOG=$INSTANCE/log/checkzopemem.log
 RESTART="$INSTANCE/bin/zopectl restart"

 process=$(cat $INSTANCE/var/Z2.pid)
 size=$(ps h -o vsize $process)
 date +"%Y/%m/%d %H:%M:%S %Z instance $INSTANCE, process $process, size $size" >>$LOG
 if [ $size -gt $MAXSIZE ]; then
   date +'%Y/%m/%d %H:%M:%S %Z '$INSTANCE' has exceeded '$MAXSIZE' limit, restarting' >>$LOG
 #  echo; echo 'recent hits:'; tail $INSTANCE/log/Z2.log | tee >>$LOG
 #  echo; echo 'recent events:'; tail -100 $INSTANCE/log/event.log | tee >>$LOG
 #  echo
   $RESTART
 fi

Issues

• this depends on Z2.pid being accurate, which may not always be true. Just killing the biggest python process might be more robust in practice.
• if zope exhausts memory between checks, this script may not be able to run.

comments:

... --simon, Thu, 25 Aug 2005 08:21:55 -0700 reply
For stopping quick blow-ups, maybe running this in a while true loop would be worthwhile.