You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Mike Clarke <mi...@mystyleit.com> on 2009/11/17 17:55:29 UTC
log4j-dev@logging.apache.org
Hello,
I work with a application that generates huge text files that are not
XML based whatsoever. There is a structure but nothing that Chainsaw
can read.
What I would like to do is to use chainsaw to read these files but I'm
not sure how to proceed.
(I've done a ton of searching but still am just spinning my wheels)
Ideally I would like to just create some short of filter using regular
expressions that would enable Chainsaw to read and parse the files.
Q1: Is this possible/and is this documented enough that I can make it
happen? Please send links!
Q2: If I can't just create some sort of config file for Chainsaw, can
I extend Chainsaw and add my own receiver.
(My receiver being my code that I tell how to read the logs)
If so, links please, can't find anything on how to get started with this!
I'm not looking to do anything across sockets, this is entirely for
offline reading.
Q3: Failing on Q1 and Q2 I could write a standalone converter to
convert my text logs got log4j but that seems very ugly.
Any ideas/links and tips would be very, very appreciated. Currently I
work with a group who reads these huge logs text editors and it sucks!
Thanks,
Mike Clarke
http://mystyleit.com
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: log4j-dev@logging.apache.org
Posted by Scott Deboy <sc...@gmail.com>.
The easiest way to get going is to define an xml configuration file that
contains the LogFilePatternReceiver configuration, and specify the URL to
that file as the auto config URL in Chainsaw's application-wide preferences
panel.
You can see an example xml configuration file from inside Chainsaw - go to
the Welcome tab and click the 'view example receiver configuration' button.
Copy this example configuration (ctrl-a, ctrl-c should work), remove all of
the 'plugin' nodes except LogFilePatternReceiver, and modify the parameters
to match what you need.
Then, specify the -URL- to this file in the automatic configuration URL
field (view/show application-wide preferences) and restart Chainsaw.
The 'logFormat' param uses the fields described in the receiver JavaDoc.
The 'filterExpression' param (this param isn't defined in the example but
you can add it if you need to) uses the expression syntax built into
Chainsaw (go to help/tutorial for a description of this syntax).
Hope that helps.
Scott
On Tue, Nov 17, 2009 at 10:21 AM, Mike Clarke <mi...@mystyleit.com>wrote:
> Hi Scott,
>
>
> Thanks for your response.
>
> That is exactly what I want to do but (unfortunately) I can't figure
> out from the documentation how to do that.
> In the org.apache.log4j.varia.LogFilePatternReceiver to me (I;'m more of a
> admin then a dev) it just looks like the API to the class.
>
> Then it goes on about an example (well you know what it says, it looks
> like you wrote it)
>
> I guess my real question is how to I get the example going. Is the
> example a config file example or a code example?
>
> Could you please provide some more details about how to use the
> org.apache.log4j.varia.LogFilePatternReceiver.
>
> Much Thanks,
> Mike Clarke
>
>
>
> On Tue, Nov 17, 2009 at 1:08 PM, Scott Deboy <sc...@gmail.com>
> wrote:
> > I'd suggest configuring Chainsaw with a LogFilePatternReceiver (or
> > VFSLogFilePatternReceiver if you need to tail the log files or get to log
> > files accessible via Jakarta Commons-VFS supported file systems).
> >
> > The JavaDoc for LogFilePatternReceiver is available from inside Chainsaw
> -
> > help/Receiver Javadoc.
> >
> > A few things you can do with the receiver:
> > - define a custom expression (logFormat) which will define how your log
> > file is parsed. It could be as simple as 'MSG' or it could include
> custom
> > properties (which will end up as separate columns in the table) - see the
> > receiver javadoc for information on how to define the logFormat.
> > - define a 'filterExpression' param (an ExpressionFilter, which supports
> > regular expressions via the 'like' keyword) which determines which log
> file
> > entries will be processed (ignoring all other entries)
> > - the svn HEAD version of the receiver includes an 'appendNonMatches'
> param
> > which will also include (with an 'unknown' logger) any rows which don't
> > match the logFormat expression
> >
> > With Chainsaw itself, you can enable or disable cyclic buffering on a
> > per-tab basis - if you turn off cyclic buffering or make it a large size,
> > I'd suggest downloading Chainsaw and modify the script to increase your
> heap
> > size).
> >
> > I also recently added a handy 'marker' feature - double-click or ctrl-f2
> to
> > toggle the marker on or off for a row..when you toggle a marker on, it
> > creates an editable column that you can use in expressions to filter and
> > search, and you can navigate between markers via f2/shift f2.
> >
> > If you save the events, the markers are also saved, so you can forward a
> log
> > to someone else and they can see your 'notes' you've added to the log.
> >
> > I use Chainsaw to parse 50MB log files and it works pretty well..of
> course,
> > I've spent a lot of time recently on the receiver and Chainsaw itself, so
> > svn HEAD will be a lot better than the WebStart/bundle download version.
> >
> > Hope that helps
> >
> > Scott
> >
> >
> > On Tue, Nov 17, 2009 at 8:55 AM, Mike Clarke <mi...@mystyleit.com>
> > wrote:
> >>
> >> Hello,
> >>
> >> I work with a application that generates huge text files that are not
> >> XML based whatsoever. There is a structure but nothing that Chainsaw
> >> can read.
> >>
> >> What I would like to do is to use chainsaw to read these files but I'm
> >> not sure how to proceed.
> >> (I've done a ton of searching but still am just spinning my wheels)
> >>
> >>
> >> Ideally I would like to just create some short of filter using regular
> >> expressions that would enable Chainsaw to read and parse the files.
> >> Q1: Is this possible/and is this documented enough that I can make it
> >> happen? Please send links!
> >>
> >>
> >> Q2: If I can't just create some sort of config file for Chainsaw, can
> >> I extend Chainsaw and add my own receiver.
> >> (My receiver being my code that I tell how to read the logs)
> >> If so, links please, can't find anything on how to get started with
> this!
> >>
> >> I'm not looking to do anything across sockets, this is entirely for
> >> offline reading.
> >>
> >>
> >>
> >> Q3: Failing on Q1 and Q2 I could write a standalone converter to
> >> convert my text logs got log4j but that seems very ugly.
> >>
> >>
> >>
> >>
> >> Any ideas/links and tips would be very, very appreciated. Currently I
> >> work with a group who reads these huge logs text editors and it sucks!
> >>
> >>
> >> Thanks,
> >> Mike Clarke
> >> http://mystyleit.com
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> >> For additional commands, e-mail: log4j-dev-help@logging.apache.org
> >>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>
>
Re: log4j-dev@logging.apache.org
Posted by Mike Clarke <mi...@mystyleit.com>.
Hi Scott,
Thanks for your response.
That is exactly what I want to do but (unfortunately) I can't figure
out from the documentation how to do that.
In the org.apache.log4j.varia.LogFilePatternReceiver to me (I;'m more of a
admin then a dev) it just looks like the API to the class.
Then it goes on about an example (well you know what it says, it looks
like you wrote it)
I guess my real question is how to I get the example going. Is the
example a config file example or a code example?
Could you please provide some more details about how to use the
org.apache.log4j.varia.LogFilePatternReceiver.
Much Thanks,
Mike Clarke
On Tue, Nov 17, 2009 at 1:08 PM, Scott Deboy <sc...@gmail.com> wrote:
> I'd suggest configuring Chainsaw with a LogFilePatternReceiver (or
> VFSLogFilePatternReceiver if you need to tail the log files or get to log
> files accessible via Jakarta Commons-VFS supported file systems).
>
> The JavaDoc for LogFilePatternReceiver is available from inside Chainsaw -
> help/Receiver Javadoc.
>
> A few things you can do with the receiver:
> - define a custom expression (logFormat) which will define how your log
> file is parsed. It could be as simple as 'MSG' or it could include custom
> properties (which will end up as separate columns in the table) - see the
> receiver javadoc for information on how to define the logFormat.
> - define a 'filterExpression' param (an ExpressionFilter, which supports
> regular expressions via the 'like' keyword) which determines which log file
> entries will be processed (ignoring all other entries)
> - the svn HEAD version of the receiver includes an 'appendNonMatches' param
> which will also include (with an 'unknown' logger) any rows which don't
> match the logFormat expression
>
> With Chainsaw itself, you can enable or disable cyclic buffering on a
> per-tab basis - if you turn off cyclic buffering or make it a large size,
> I'd suggest downloading Chainsaw and modify the script to increase your heap
> size).
>
> I also recently added a handy 'marker' feature - double-click or ctrl-f2 to
> toggle the marker on or off for a row..when you toggle a marker on, it
> creates an editable column that you can use in expressions to filter and
> search, and you can navigate between markers via f2/shift f2.
>
> If you save the events, the markers are also saved, so you can forward a log
> to someone else and they can see your 'notes' you've added to the log.
>
> I use Chainsaw to parse 50MB log files and it works pretty well..of course,
> I've spent a lot of time recently on the receiver and Chainsaw itself, so
> svn HEAD will be a lot better than the WebStart/bundle download version.
>
> Hope that helps
>
> Scott
>
>
> On Tue, Nov 17, 2009 at 8:55 AM, Mike Clarke <mi...@mystyleit.com>
> wrote:
>>
>> Hello,
>>
>> I work with a application that generates huge text files that are not
>> XML based whatsoever. There is a structure but nothing that Chainsaw
>> can read.
>>
>> What I would like to do is to use chainsaw to read these files but I'm
>> not sure how to proceed.
>> (I've done a ton of searching but still am just spinning my wheels)
>>
>>
>> Ideally I would like to just create some short of filter using regular
>> expressions that would enable Chainsaw to read and parse the files.
>> Q1: Is this possible/and is this documented enough that I can make it
>> happen? Please send links!
>>
>>
>> Q2: If I can't just create some sort of config file for Chainsaw, can
>> I extend Chainsaw and add my own receiver.
>> (My receiver being my code that I tell how to read the logs)
>> If so, links please, can't find anything on how to get started with this!
>>
>> I'm not looking to do anything across sockets, this is entirely for
>> offline reading.
>>
>>
>>
>> Q3: Failing on Q1 and Q2 I could write a standalone converter to
>> convert my text logs got log4j but that seems very ugly.
>>
>>
>>
>>
>> Any ideas/links and tips would be very, very appreciated. Currently I
>> work with a group who reads these huge logs text editors and it sucks!
>>
>>
>> Thanks,
>> Mike Clarke
>> http://mystyleit.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: log4j-dev@logging.apache.org
Posted by Scott Deboy <sc...@gmail.com>.
I'd suggest configuring Chainsaw with a LogFilePatternReceiver (or
VFSLogFilePatternReceiver if you need to tail the log files or get to log
files accessible via Jakarta Commons-VFS supported file systems).
The JavaDoc for LogFilePatternReceiver is available from inside Chainsaw -
help/Receiver Javadoc.
A few things you can do with the receiver:
- define a custom expression (logFormat) which will define how your log
file is parsed. It could be as simple as 'MSG' or it could include custom
properties (which will end up as separate columns in the table) - see the
receiver javadoc for information on how to define the logFormat.
- define a 'filterExpression' param (an ExpressionFilter, which supports
regular expressions via the 'like' keyword) which determines which log file
entries will be processed (ignoring all other entries)
- the svn HEAD version of the receiver includes an 'appendNonMatches' param
which will also include (with an 'unknown' logger) any rows which don't
match the logFormat expression
With Chainsaw itself, you can enable or disable cyclic buffering on a
per-tab basis - if you turn off cyclic buffering or make it a large size,
I'd suggest downloading Chainsaw and modify the script to increase your heap
size).
I also recently added a handy 'marker' feature - double-click or ctrl-f2 to
toggle the marker on or off for a row..when you toggle a marker on, it
creates an editable column that you can use in expressions to filter and
search, and you can navigate between markers via f2/shift f2.
If you save the events, the markers are also saved, so you can forward a log
to someone else and they can see your 'notes' you've added to the log.
I use Chainsaw to parse 50MB log files and it works pretty well..of course,
I've spent a lot of time recently on the receiver and Chainsaw itself, so
svn HEAD will be a lot better than the WebStart/bundle download version.
Hope that helps
Scott
On Tue, Nov 17, 2009 at 8:55 AM, Mike Clarke <mi...@mystyleit.com>wrote:
> Hello,
>
> I work with a application that generates huge text files that are not
> XML based whatsoever. There is a structure but nothing that Chainsaw
> can read.
>
> What I would like to do is to use chainsaw to read these files but I'm
> not sure how to proceed.
> (I've done a ton of searching but still am just spinning my wheels)
>
>
> Ideally I would like to just create some short of filter using regular
> expressions that would enable Chainsaw to read and parse the files.
> Q1: Is this possible/and is this documented enough that I can make it
> happen? Please send links!
>
>
> Q2: If I can't just create some sort of config file for Chainsaw, can
> I extend Chainsaw and add my own receiver.
> (My receiver being my code that I tell how to read the logs)
> If so, links please, can't find anything on how to get started with this!
>
> I'm not looking to do anything across sockets, this is entirely for
> offline reading.
>
>
>
> Q3: Failing on Q1 and Q2 I could write a standalone converter to
> convert my text logs got log4j but that seems very ugly.
>
>
>
>
> Any ideas/links and tips would be very, very appreciated. Currently I
> work with a group who reads these huge logs text editors and it sucks!
>
>
> Thanks,
> Mike Clarke
> http://mystyleit.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>
>