error log analyzer/reporter

[Jonathan Swartz <sw...@pobox.com>]

We want to periodically analyze our apache error logs, and send out  
email reports with summaries of new errors that have occurred since  
the last report. Ideally, similar errors would be combined into  
groups, and reported in order of frequency.

This seems like a common task (something I've wanted in some form at  
every job :)) and so I'm wondering, before I write it yet again, if  
there is a good package out there.

I did some googling, but found that most tools are geared towards  
processing access log statistics rather than error logs. We've  
already got that part handled by omniture.

We just started using smolder to aggregate test results - something  
like that for error logs would be fantastic.

TIA for any pointers!

Jon

Re: error log analyzer/reporter

[E R <pc...@gmail.com>]

I have a need to develop such an error-log analyzer but with real-time
analysis. The approach I am currently considering is to implement it
as a specialized web server so it can asynchronously process the log
messages and respond to requests for gathered statistics. I'd like to
write it all in perl, but I would consider other solutions if they are
easier to maintain.

Let me know if you would like to collaborate on this.

ER

Re: error log analyzer/reporter

[John ORourke <jo...@o-rourke.org>]

Jonathan Swartz wrote:
> We want to periodically analyze our apache error logs, and send out 
> email reports with 
> TIA for any pointers!

Seconded, that'd be truly useful.  I'm currently working out how best to 
do a weekly 'sweep' for recurring or unusual errors in the logs - things 
that may not affect users but could invoke the 'next tuesday' rule.  
(see http://blogs.msdn.com/larryosterman/archive/2004/03/30/104165.aspx )

John

Re: error log analyzer/reporter

[Perrin Harkins <pe...@elem.com>]

On Nov 14, 2007 1:17 PM, Jonathan Swartz <sw...@pobox.com> wrote:
> We want to periodically analyze our apache error logs, and send out
> email reports with summaries of new errors that have occurred since
> the last report. Ideally, similar errors would be combined into
> groups, and reported in order of frequency.

I could swear I asked for this before on the list and got suggestions,
but I can't find it in the archive.  I think most people use existing
monitoring tools like Nagios for this.  LogWatch is also a popular one
that comes with most Linux systems.

- Perrin