You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Joseph Husby <hu...@umn.edu> on 2018/01/19 15:32:18 UTC
Chainsaw log-parsing classes
Hello!
I’m working on a project where we need to take log4j2 output and (offline)
load it into a Mongo database. We can't use a mongo appender because the
database is not available where the software is running. We could use the
log4j2's JsonLayout to have an easily parsable log format, but would prefer
not to as it makes the logs less human readable (especially timestamps).
Obviously Chainsaw has the capability to parse standard log4j output, but
is any of that exposed so we could use those classes in our application?
We don’t need the GUI component.
Thank you for any help you’re able to provide!
~ Joe ~
p.s. resending this after subscribing to the list as I don't see my
original message in the archive, so assuming it was rejected. Sorry if I'm
double messaging!
Re: Chainsaw log-parsing classes
Posted by Ralph Goers <ra...@dslextreme.com>.
If you application is running somewhere and Mongo is running somewhere else I would recommend using the Flume Appender to send the events to a Flume node that is running where Mongo is. Then have Flume write the events to Mongo.
Ralph
> On Jan 19, 2018, at 8:32 AM, Joseph Husby <hu...@umn.edu> wrote:
>
> Hello!
>
>
>
> I’m working on a project where we need to take log4j2 output and (offline)
> load it into a Mongo database. We can't use a mongo appender because the
> database is not available where the software is running. We could use the
> log4j2's JsonLayout to have an easily parsable log format, but would prefer
> not to as it makes the logs less human readable (especially timestamps).
>
>
> Obviously Chainsaw has the capability to parse standard log4j output, but
> is any of that exposed so we could use those classes in our application?
> We don’t need the GUI component.
>
>
>
> Thank you for any help you’re able to provide!
>
>
> ~ Joe ~
>
> p.s. resending this after subscribing to the list as I don't see my
> original message in the archive, so assuming it was rejected. Sorry if I'm
> double messaging!
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Chainsaw log-parsing classes
Posted by Scott Deboy <sc...@gmail.com>.
Yeah - good idea.
Here's the code Chainsaw uses to grab a log4j XML config's
FileAppender definition, including the PatternLayout, and build what's
needed to process the log messages going in to the target file.
BTW this is also how the log4j2 Advertiser integrates with Chainsaw.
Map entries =
LogFilePatternLayoutBuilder.getAppenderConfiguration(log4jConfigFile);
for (Iterator iter =
entries.entrySet().iterator();iter.hasNext();) {
try {
Map.Entry entry = (Map.Entry)iter.next();
String name = (String) entry.getKey();
Map values = (Map) entry.getValue();
//values: conversion, file
String conversionPattern =
values.get("conversion").toString();
File file = new File(values.get("file").toString());
URL fileURL = file.toURI().toURL();
String timestampFormat =
LogFilePatternLayoutBuilder.getTimeStampFormat(conversionPattern);
String receiverPattern =
LogFilePatternLayoutBuilder.getLogFormatFromPatternLayout(conversionPattern);
VFSLogFilePatternReceiver fileReceiver = new
VFSLogFilePatternReceiver();
fileReceiver.setName(name);
fileReceiver.setAutoReconnect(true);
fileReceiver.setContainer(LogUI.this);
fileReceiver.setAppendNonMatches(true);
fileReceiver.setFileURL(fileURL.toURI().toString());
fileReceiver.setTailing(true);
fileReceiver.setLogFormat(receiverPattern);
fileReceiver.setTimestampFormat(timestampFormat);
fileReceiver.setThreshold(Level.TRACE);
pluginRegistry.addPlugin(fileReceiver);
fileReceiver.activateOptions();
Scott
On 1/20/18, Mikael Ståldal <mi...@apache.org> wrote:
> Maybe we could take some code from Chainsaw and implement a
> LogEventParser for Log4j 2?
>
> https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser/TextLogEventParser.java
>
>
> On 2018-01-19 17:00, Scott Deboy wrote:
>> Here's info on how you can use the same Receiver Chainsaw uses in your
>> own
>> application - note this is not built in to log4j2:
>>
>> https://stackoverflow.com/questions/47605483/log4j-unstructured-log-parser-in-java-or-scala?rq=1
>>
>> Scott
>>
>> On Jan 19, 2018 7:32 AM, "Joseph Husby" <hu...@umn.edu> wrote:
>>
>> Hello!
>>
>>
>>
>> I’m working on a project where we need to take log4j2 output and
>> (offline)
>> load it into a Mongo database. We can't use a mongo appender because the
>> database is not available where the software is running. We could use
>> the
>> log4j2's JsonLayout to have an easily parsable log format, but would
>> prefer
>> not to as it makes the logs less human readable (especially timestamps).
>>
>>
>> Obviously Chainsaw has the capability to parse standard log4j output, but
>> is any of that exposed so we could use those classes in our application?
>> We don’t need the GUI component.
>>
>>
>>
>> Thank you for any help you’re able to provide!
>>
>>
>> ~ Joe ~
>>
>> p.s. resending this after subscribing to the list as I don't see my
>> original message in the archive, so assuming it was rejected. Sorry if
>> I'm
>> double messaging!
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Chainsaw log-parsing classes
Posted by Mikael Ståldal <mi...@apache.org>.
Maybe we could take some code from Chainsaw and implement a
LogEventParser for Log4j 2?
https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser/TextLogEventParser.java
On 2018-01-19 17:00, Scott Deboy wrote:
> Here's info on how you can use the same Receiver Chainsaw uses in your own
> application - note this is not built in to log4j2:
>
> https://stackoverflow.com/questions/47605483/log4j-unstructured-log-parser-in-java-or-scala?rq=1
>
> Scott
>
> On Jan 19, 2018 7:32 AM, "Joseph Husby" <hu...@umn.edu> wrote:
>
> Hello!
>
>
>
> I’m working on a project where we need to take log4j2 output and (offline)
> load it into a Mongo database. We can't use a mongo appender because the
> database is not available where the software is running. We could use the
> log4j2's JsonLayout to have an easily parsable log format, but would prefer
> not to as it makes the logs less human readable (especially timestamps).
>
>
> Obviously Chainsaw has the capability to parse standard log4j output, but
> is any of that exposed so we could use those classes in our application?
> We don’t need the GUI component.
>
>
>
> Thank you for any help you’re able to provide!
>
>
> ~ Joe ~
>
> p.s. resending this after subscribing to the list as I don't see my
> original message in the archive, so assuming it was rejected. Sorry if I'm
> double messaging!
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Chainsaw log-parsing classes
Posted by Scott Deboy <sc...@gmail.com>.
Here's info on how you can use the same Receiver Chainsaw uses in your own
application - note this is not built in to log4j2:
https://stackoverflow.com/questions/47605483/log4j-unstructured-log-parser-in-java-or-scala?rq=1
Scott
On Jan 19, 2018 7:32 AM, "Joseph Husby" <hu...@umn.edu> wrote:
Hello!
I’m working on a project where we need to take log4j2 output and (offline)
load it into a Mongo database. We can't use a mongo appender because the
database is not available where the software is running. We could use the
log4j2's JsonLayout to have an easily parsable log format, but would prefer
not to as it makes the logs less human readable (especially timestamps).
Obviously Chainsaw has the capability to parse standard log4j output, but
is any of that exposed so we could use those classes in our application?
We don’t need the GUI component.
Thank you for any help you’re able to provide!
~ Joe ~
p.s. resending this after subscribing to the list as I don't see my
original message in the archive, so assuming it was rejected. Sorry if I'm
double messaging!
Re: Chainsaw log-parsing classes
Posted by Mikael Ståldal <mi...@apache.org>.
FWIW, Log4j 2's log4j-core has an API to parse the output of JsonLayout,
XmlLayout and YamlLayout:
https://github.com/apache/logging-log4j2/tree/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser
On 2018-01-19 16:32, Joseph Husby wrote:
> Hello!
>
>
>
> I’m working on a project where we need to take log4j2 output and (offline)
> load it into a Mongo database. We can't use a mongo appender because the
> database is not available where the software is running. We could use the
> log4j2's JsonLayout to have an easily parsable log format, but would prefer
> not to as it makes the logs less human readable (especially timestamps).
>
>
> Obviously Chainsaw has the capability to parse standard log4j output, but
> is any of that exposed so we could use those classes in our application?
> We don’t need the GUI component.
>
>
>
> Thank you for any help you’re able to provide!
>
>
> ~ Joe ~
>
> p.s. resending this after subscribing to the list as I don't see my
> original message in the archive, so assuming it was rejected. Sorry if I'm
> double messaging!
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org