A tool in SOFTWARE_HOME/bin for analyzing your trace.log. Very useful for eg finding hanging or server-crashing requests. Zope 3's equivalent is called tracereport.
Here is the --help output from the zope 2.10.3 version. Note "debug.log" is an older name for trace.log:
Usage: /zope/bin/requestprofiler.py filename1 [filename2 ...]
[--cumulative | --detailed | [--timed --resolution=seconds]]
[--sort=spec]
[--top=n]
[--verbose]
[--today | [--start=date] [--end=date] | --daysago=n ]
[--writestats=filename | --readstats=filename]
[--urlfocus=url]
[--urlfocustime=seconds]
[--help]
Provides a profile of one or more Zope "-M" request log files.
Reports are of four types: cumulative, detailed, timed, or urlfocus. The default is cumulative. Data is taken from one or more Zope detailed request logs (-M logs, aka 'big M' logs) or from a preprocessed statistics file.
For cumulative reports, each line in the profile indicates information about a Zope method (URL) collected via a detailed request log.
For detailed reports, each line in the profile indicates information about a single request.
For timed reports, each line in the profile indicates informations about the number of requests and the number of requests/second for a period of time.
For urlfocus reports, ad-hoc information about requests surrounding the specified url is given.
Each 'filename' is a path to a '-M' log that contains detailed request data. Multiple input files can be analyzed at the same time by providing the path to each file. (Analyzing multiple big M log files at once is useful if you have more than one Zope client on a single machine and you'd like to get an overview of all Zope activity on that machine).
If you wish to make multiple analysis runs against the same input data, you may want to use the --writestats option. The --writestats option creates a file which holds preprocessed data representing the specfified input files. Subsequent runs (for example with a different sort spec) will be much faster if the --readstats option is used to specify a preprocessed stats file instead of actual input files because the logfile parse step is skipped.
If a 'sort' value is specified, sort the profile info by the spec. The sort order is descending unless indicated. The default cumulative sort spec is 'total'. The default detailed sort spec is 'start'.
For cumulative reports, the following sort specs are accepted:
'hits' -- the number of hits against the method 'hangs' -- the number of unfinished requests to the method 'max' -- the maximum time in secs taken by a request to this method 'min' -- the minimum time in secs taken by a request to this method 'mean' -- the mean time in secs taken by a request to this method 'median' -- the median time in secs taken by a request to this method 'total' -- the total time in secs across all requests to this method 'url' -- the URL/method name (ascending)
For detailed (non-cumulative) reports, the following sort specs are accepted:
'start' -- the start time of the request to ZServer (ascending) 'win' -- the num of secs ZServer spent waiting for input from client 'wout' -- the secs ZServer spent waiting for output from ZPublisher 'wend' -- the secs ZServer spent sending data to the client 'total' -- the secs taken for the request from begin to end 'endstage' -- the last successfully completed request stage (B, I, A, E) 'osize' -- the size in bytes of output provided by ZPublisher 'httpcode' -- the HTTP response code provided by ZPublisher (ascending) 'active' -- total num of requests pending at the end of this request 'url' -- the URL/method name (ascending)
For timed and urlfocus reports, there are no sort specs allowed.
If the 'top' argument is specified, only report on the top 'n' entries in the profile (as per the sort). The default is to show all data in the profile.
If the 'verbose' argument is specified, do not trim url to fit into 80 cols.
If the 'today' argument is specified, limit results to hits received today.
If the 'daysago' argument is specified, limit results to hits received n days ago.
The 'resolution' argument is used only for timed reports and specifies the number of seconds between consecutive lines in the report (default is 60 seconds).
The 'urlfocustime' argument is used only for urlfocus reports and specifies the number of seconds to target before and after the URL provided in urlfocus mode. (default is 10 seconds).
If the 'start' argument is specified in the form 'DD/MM/YYYY HH:MM:SS' (UTC), limit results to hits received after this date/time.
If the 'end' argument is specified in the form 'DD/MM/YYYY HH:MM:SS' (UTC), limit results to hits received before this date/time.
'start' and 'end' arguments are not honored when request stats are obtained via the --readstats argument.
Examples:
/zope/bin/requestprofiler.py debug.log
Show cumulative report statistics for information in the file 'debug.log', by default sorted by 'total'.
/zope/bin/requestprofiler.py debug.log --detailed
Show detailed report statistics sorted by 'start' (by default).
/zope/bin/requestprofiler.py debug.log debug2.log --detailed
Show detailed report statistics for both logs sorted by 'start' (by default).
/zope/bin/requestprofiler.py debug.log --cumulative --sort=mean --today --verbose
Show cumulative report statistics sorted by mean for entries in the log which happened today, and do not trim the URL in the resulting report.
/zope/bin/requestprofiler.py debug.log --cumulative --sort=mean --daysago=3 --verbose
Show cumulative report statistics sorted by mean for entries in the log which happened three days ago, and do not trim the URL in the resulting report.
/zope/bin/requestprofiler.py debug.log --urlfocus='/manage_main' --urlfocustime=60
Show 'urlfocus' report which displays statistics about requests surrounding the invocation of '/manage_main'. Focus on the time periods 60 seconds before and after each invocation of the '/manage_main' URL.
/zope/bin/requestprofiler.py debug.log --detailed --start='2001/05/10 06:00:00' --end='2001/05/11 23:00:00'
Show detailed report statistics for entries in 'debug.log' which begin after 6am UTC on May 10, 2001 and which end before 11pm UTC on May 11, 2001.
/zope/bin/requestprofiler.py debug.log --timed --resolution=300 --start='2001/05/10 06:00:00' --end='2001/05/11 23:00:00'
Show timed report statistics for entries in the log for one day with a resolution of 5 minutes
/zope/bin/requestprofiler.py debug.log --top=100 --sort=max
Show cumulative report of the the 'top' 100 methods sorted by maximum elapsed time.
/zope/bin/requestprofiler.py debug.log debug2.log --writestats='requests.stat'
Write stats file for debug.log and debug2.log into 'requests.stat' and show default report.
/zope/bin/requestprofiler.py --readstats='requests.stat' --detailed
Read from 'requests.stat' stats file (instead of actual -M log files) and show detailed report against this data.
More examples:
/zope/bin/requestprofiler.py trace.log --verbose --start='YYYY/MM/DD 00:00:00' --end='YYYY/MM/DD 23:59:59' --detailed --sort=total
Show all requests on this day, slowest first (including transmission time).
/zope/bin/requestprofiler.py trace.log --verbose --start='YYYY/MM/DD 00:00:00' --end='YYYY/MM/DD 23:59:59' --detailed --sort=wout
Show all requests on this day, slowest first (counting zope processing time only).