You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Joan <as...@gmail.com> on 2019/05/21 13:11:36 UTC
Configuring log format for console output (catalina.out)
Hi all,
I am trying to forward the logs generated on a server to a centralized log
collector (using rsyslog), one of the features I would like to use the
ability to group messages by using their timestamp as the start of the
message.
By reading the documentation at
https://tomcat.apache.org/tomcat-9.0-doc/logging.html#Console, I can't see
the proper way to get a common format (preferably with timestamp), or ar
tleast save the outputs into three different types in different files?
Some of the logs are displayed like this (severity in brackets but no
timestamp)
>
> [WARN] Web application uses
> org.apache.axis2.transport.http.AxisAdminServlet; please update web.xml to
> use org.apache.axis2.webapp.AxisAdminServlet instead
> [INFO] Clustering has been disabled
> [INFO] Deploying module: script-1.6.2 -
> file:/var/lib/tomcat8/webapps/WS_Revisiones/WEB-INF/modules/scripting-1.6.2.mar
>
Stack traces are displayed like this (no severity and no timestamp)
> MultiException stack 1 of 2
> java.lang.NoClassDefFoundError: javax/activation/DataSource
> at java.base/java.lang.Class.getDeclaredConstructors0(Native
> Method)
> at
> java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
> at
> java.base/java.lang.Class.getDeclaredConstructors(Class.java:2357)
> at org.jvnet.hk2.internal.Utilities$3.run(Utilities.java:1306)
> at org.jvnet.hk2.internal.Utilities$3.run(Utilities.java:1302)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
>
Also some other are like this (timestamp with ms + severity + context in
brackets
> 21-May-2019 13:03:22.474 INFO [localhost-startStop-1]
> org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned
> for TLDs yet contained no TLDs. Enable debug logging for this logger for a
> complete list of JARs that were scanned but no TLDs were found in them.
> Skipping unneeded JARs during scanning can improve startup time and JSP
> compilation time.
>
Re: Configuring log format for console output (catalina.out)
Posted by Joan <as...@gmail.com>.
Thanks Guido,
I hope that with the rsyslog magic (see
startmsg.regex="^[[:digit:]]{2}-[[:alpha:]]{3}-[[:digit:]]{4}
[[:digit:]]{1,2}:[[:digit:]]{1,2}:[[:digit:]]{1,2}\\.[[:digit:]]{3}" for
the reference), I will be able to group this messages, otherwise I'll jump
into your solution.
Joan
Missatge de Jäkel, Guido <G....@dnb.de> del dia dc., 22 de maig 2019 a
les 8:55:
> Dear Joan,
>
> by use of the common scripts, the file catalina.out will contain the
> console output (stdout/tderr) of the *JVM* process and -- if not configured
> in another way -- of the applications. Therefore, you have to deal with the
> features of output formatting of current JVMs. Or -- as me -- you might
> pipe this file descriptors through a tiny script that will prepend a
> timestamp in a format of your choice. This will have the advantage that it
> also handle "console" output of Java application; despite of the fact that
> using stdout/stderr instead of a java logging mechanism is a very bad style.
>
> If you don't need to process thousands of lines per second, a simple shell
> script may do the job:
>
> while read line; do echo "`date -Ins` $line"; done
>
> To avoid "double-stamping", you may add an heuristic check of the incoming
> line; please adjust the RegExpr to your neeeds
>
> while read line; do [[ ! "$line" =~ ^\d\d\d\d ]] && echo -n
> "[`date -Ins`] "; echo $line; done
>
>
> Guido
>
RE: Configuring log format for console output (catalina.out)
Posted by Jäkel,
Guido <G....@dnb.de>.
Dear Joan,
by use of the common scripts, the file catalina.out will contain the console output (stdout/tderr) of the *JVM* process and -- if not configured in another way -- of the applications. Therefore, you have to deal with the features of output formatting of current JVMs. Or -- as me -- you might pipe this file descriptors through a tiny script that will prepend a timestamp in a format of your choice. This will have the advantage that it also handle "console" output of Java application; despite of the fact that using stdout/stderr instead of a java logging mechanism is a very bad style.
If you don't need to process thousands of lines per second, a simple shell script may do the job:
while read line; do echo "`date -Ins` $line"; done
To avoid "double-stamping", you may add an heuristic check of the incoming line; please adjust the RegExpr to your neeeds
while read line; do [[ ! "$line" =~ ^\d\d\d\d ]] && echo -n "[`date -Ins`] "; echo $line; done
Guido
Re: Configuring log format for console output (catalina.out)
Posted by Joan <as...@gmail.com>.
Thanks a lot for your answer Chris
> > I am trying to forward the logs generated on a server to a
> > centralized log collector (using rsyslog), one of the features I
> > would like to use the ability to group messages by using their
> > timestamp as the start of the message. By reading the documentation
> > at https://tomcat.apache.org/tomcat-9.0-doc/logging.html#Console, I
> > can't see the proper way to get a common format (preferably with
> > timestamp), or ar tleast save the outputs into three different
> > types in different files?
>
> I think you are being confused by the fact that the default log
> pattern contains a newline character. For example, these are the first
> two entries in my local catalina.out file:
>
> Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init
> INFO: Initializing ProtocolHandler ["ajp-nio-8215"]
> Apr 12, 2019 1:55:03 PM org.apache.tomcat.util.net.NioSelectorPool
> getSharedSelector
> INFO: Using a shared selector for servlet write/read
> Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init
>
> (Apologies for the word-wrapping)
>
> The first entry is [Apr 12, 2019 1:55:03 PM] and the text ends with
> ["ajp-nio-8215"]
>
> The second entry has the same timestmp and the text ends with
> "AbstractProtocol init"
>
> Stack traces are always impossible to handle when looking at the
> resulting file, since they almost always span multiple lines of text
> - -- that is, they have embedded newlines in the message.
>
> Are you asking about how to make sense of the log file, how to send
> log messages to a central server, or how to coalesce log messages by
> timestamp?
>
Rsyslog allows to read multiline messages by matching a regular expression
(see
https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#startmsg-regex),
this allow the messages to be forwarded to a third party (graylog,
logstash, ..) as a a single message.
So my intention would be to be able to make all the messages to start with
a timestamp (or a defined format)
After your message I undertand that this is a single message (I didn't
realize it) and all the messages in catalina.out are obeying the same
format. Being the next lines a single message, am I right?
19-May-2019 06:25:20.937 INFO [localhost-startStop-1]
> org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned
> for TLDs yet contained no TLDs. Enable debug logging for this logger for a
> comple
> te list of JARs that were scanned but no TLDs were found in them. Skipping
> unneeded JARs during scanning can improve startup time and JSP compilation
> time.
> [ERROR] Unable to find config file. Creating new servlet engine config
> file: /WEB-INF/server-config.wsdd
> [WARN] Web application uses
> org.apache.axis2.transport.http.AxisAdminServlet; please update web.xml to
> use org.apache.axis2.webapp.AxisAdminServlet instead
> [WARN] Unable to load manifest for StAX implementation at jrt:/java.xml/
> [INFO] Clustering has been disabled
>
Thanks,
Joan
>
>
Re: Configuring log format for console output (catalina.out)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Joan,
On 5/21/19 09:11, Joan wrote:
> Hi all,
>
> I am trying to forward the logs generated on a server to a
> centralized log collector (using rsyslog), one of the features I
> would like to use the ability to group messages by using their
> timestamp as the start of the message. By reading the documentation
> at https://tomcat.apache.org/tomcat-9.0-doc/logging.html#Console, I
> can't see the proper way to get a common format (preferably with
> timestamp), or ar tleast save the outputs into three different
> types in different files?
I think you are being confused by the fact that the default log
pattern contains a newline character. For example, these are the first
two entries in my local catalina.out file:
Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-nio-8215"]
Apr 12, 2019 1:55:03 PM org.apache.tomcat.util.net.NioSelectorPool
getSharedSelector
INFO: Using a shared selector for servlet write/read
Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init
(Apologies for the word-wrapping)
The first entry is [Apr 12, 2019 1:55:03 PM] and the text ends with
["ajp-nio-8215"]
The second entry has the same timestmp and the text ends with
"AbstractProtocol init"
Stack traces are always impossible to handle when looking at the
resulting file, since they almost always span multiple lines of text
- -- that is, they have embedded newlines in the message.
Are you asking about how to make sense of the log file, how to send
log messages to a central server, or how to coalesce log messages by
timestamp?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlzkEO8ACgkQHPApP6U8
pFg8EA/+KsH4JGirKakL+/OxqAz1W7T7MoR2QE9Y1Tk26FL6XI+VITk11gqLegH5
yx3zwmXLS9v2XilppnSvRB9xPd073Uf4v0NPSjCt1vSY7obPHYcRSIuA43QU9Pqb
7pEgxuGn+uUifRRK+DLd6Wcd3IkFo2hfKG/YHFL5McKxqsrEvPsspWU7uLRHoKML
BR1MM5iKbsiBEJfncfyZqeGY2Yq4HOarcp//iv5bC20RgakLxlSjiWAyJqC77PKH
3dy9XaV/VyQpjEQe8/ADt1NqhF6qBVj/KcKeA41AAwL7grxX0R23ZeSQsKJeulgf
jnXcK8mHWXs4Dybu7hhUbkS47whbfW6M3H4ZhGYutsoOB58shzbVR0MUtEYlOBMg
GbpCt5xPW6NZ5LMbEDTMlQW95XBnrLdxe8dFdTo+qgEdSoe3o4mdvGUtrlgAxZlq
Ay28U+/maI84StrScDwfC96chakdaGj/4CxyFwoj+5uVeeiBbglubR1gCghqVjM7
MQWDKrTACmcEu4ZbLy1sgNtBb/xjCYRtp05JrB531E9Py7//f6qprIKjAYOBLcxF
uWDKP204C3UlC3/mQt6Ewd6QzQM0xCXPm0k+6817wf3xIaN/aST4R0kbR0OjkB6F
CfAm8Qf2FVgRHOP4zzKHvm82wjET2FJmcw38do2ki89yml1iKXg=
=UolI
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org