You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Randeep <ra...@gmail.com> on 2014/03/18 15:13:15 UTC

[Off Topic] Forwarding tomcat logs

Hi,

I have servers in Amazon Web Services Platform.

My servers are Centos 5.4

I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them

I'm using elastic load balancer with autoscaling, so, many instances can
spawn and terminate anytime. But once the instance is terminated tomcat
logs of that instance is no more available.

I was trying to forward httpd and Apache tomcat logs to a central logging
server (splunk/graylog2). But was not able to.

http://stackoverflow.com/questions/22293195/forwarding-logs-to-splunk-gralog-from-syslog-ng

Has anyone done this? What is the best way to do it?

Kindly advise.

-- Randeep

Re: [Off Topic] Forwarding tomcat logs

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Randeep,

On 3/18/14, 10:13 AM, Randeep wrote:
> I have servers in Amazon Web Services Platform.
> 
> My servers are Centos 5.4
> 
> I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
> 
> I'm using elastic load balancer with autoscaling

Each server is running httpd+tomcat? Do you actually need httpd in
there for anything? If not, remove it and simplify your life.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTKJ7hAAoJEBzwKT+lPKRYo2cQAIiZL+K+YbjDFgRns8+xMmLP
OIm/nU7/4e5pXS7YTn9k0W4l0rpJns6/hPtbWNhD/5zrTdz5ZnfAsr6IuQCDoJen
dRxx1LuWXz3GjFVjqSJ0hXP7sSzPwCWZuNvfLO4eHYf1ow8wPcavS0mFr7t9QtdO
rMmssRnUPtTCBr7KGuQEf+d62guqAvweA8DOEUOfHGlBWXm3b+DuQp9qK9oGWPvF
uiuF4A+w3uWmSgA3B2Cs2X2FwM4aJ/tztGAOHekmNKimc1JYDRd60gB1iDenFc4s
JxOEjqqjqDG99NCDrwS0cjEQRPJA5G6uhaTZ6DLWJCfUgvXp2TjURpkPQjv6ssJV
PKMau+8niuAIWO+sBn4tSekNIa3hZlm5e+uoFKyO8/vUsX5nEvYKggAlN14ja7Ja
AJ/D70otmD49UoeG4QwE8cpTMRp2pOmCmggHCRnYkGxWyZWvADPs07ZD+cgF/7m0
vf89aNQzbXursX1MAM0kjTWWcb092G6JOIyUjG8j2ZIUjNOxQztJ0zkSqtyNgAd+
gVTG3USil3U6xAnod1gjJZvOW82imMc7el/d42K0dOm5fDtg6Gv4aZE5T8b8D7Du
HUqX6MdjiEpXnpDtfUdZHFDnzLjLWNf2PoG/PeBfqlOLe5UmPnLnnUCMseIn0Dkh
mF499Ym4cxYxBF33KrjD
=wQd1
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: [Off Topic] Forwarding tomcat logs

Posted by Brett Delle Grazie <br...@gmail.com>.
Hi,


On 18 March 2014 14:40, Daniel Mikusa <dm...@gopivotal.com> wrote:

> On Mar 18, 2014, at 10:13 AM, Randeep <ra...@gmail.com> wrote:
>
> > Hi,
> >
> > I have servers in Amazon Web Services Platform.
> >
> > My servers are Centos 5.4
> >
> > I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
> >
> > I'm using elastic load balancer with autoscaling, so, many instances can
> > spawn and terminate anytime. But once the instance is terminated tomcat
> > logs of that instance is no more available.
> >
> > I was trying to forward httpd
>
> ErrorLog supports sending logs to syslog.
>
>   http://httpd.apache.org/docs/2.2/mod/core.html#errorlog
>
> I'm not sure about CustomLog.  That might take a little more work.  Are
> you trying to send access logs as well?
>
> > and Apache tomcat logs
>
> First thing that comes to mind here is using Log4j and it's syslog
> appender.
>
> You can setup Tomcat to use Log4j.  Instructions here.
>
>   http://tomcat.apache.org/tomcat-7.0-doc/logging.html#Using_Log4j
>
> Then configure an appender to use SyslogAppender.
>
>
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html
>
> That should get you anything that would normally be written to a Tomcat
> log (catalina.yyyy-mm-dd.log or localhost.yyyy-mm-dd.log).  It wouldn't get
> you access logs or anything your apps are writing to System.out /
> System.err.  Again, how much are you trying to send to your syslog server?
>

An alternative is to use Logback as it includes an access log filter for
Tomcat too
https://github.com/grgrzybek/tomcat-slf4j-logback

Note that you will have to configure Logback to write to Syslog (see
logback docs: http://logback.qos.ch/documentation.html), just as you would
for Log4J.

Depending upon the syslog implementation used in CentOS 5.4 you may want to
look at alternatives for improved performance / avoidance of dropped
messages etc. For example, you could use rsyslog or nsyslog

Brett

>
> Dan
>
> > to a central logging
> > server (splunk/graylog2). But was not able to.
> >
> >
> http://stackoverflow.com/questions/22293195/forwarding-logs-to-splunk-gralog-from-syslog-ng
> >
> > Has anyone done this? What is the best way to do it?
> >
> > Kindly advise.
> >
> > -- Randeep
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: [Off Topic] Forwarding tomcat logs

Posted by Daniel Mikusa <dm...@gopivotal.com>.
On Mar 19, 2014, at 5:37 AM, Randeep <ra...@gmail.com> wrote:

> Chris,
> 
> 
> On Wed, Mar 19, 2014 at 1:01 AM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>> 
>> Dan,
>> 
>> On 3/18/14, 10:40 AM, Daniel Mikusa wrote:
>>> On Mar 18, 2014, at 10:13 AM, Randeep <ra...@gmail.com>
>>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I have servers in Amazon Web Services Platform.
>>>> 
>>>> My servers are Centos 5.4
>>>> 
>>>> I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
>> 
> 
> I'm using httpd as front end because I have a few redirection rules set.
> Also I was using httpd for handling ssl certificates.
> 
>>>> 
>>>> I'm using elastic load balancer with autoscaling, so, many
>>>> instances can spawn and terminate anytime. But once the instance
>>>> is terminated tomcat logs of that instance is no more available.
>>>> 
>>>> I was trying to forward httpd
>>> 
>>> ErrorLog supports sending logs to syslog.
>>> 
>>> http://httpd.apache.org/docs/2.2/mod/core.html#errorlog
>> 
>> +1
>> 
>> Syslog is pretty much the answer to the whole question.
>> 
> 
> I have configured my httpd.conf as below
> ErrorLog syslog:local7
> 
> And syslog.conf as
> local7.*                                                 @
> ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
> local1.*                                                 @
> ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
> 
> But I'm not getting any log in error_log or syslog central server.
> Is it possible to get logs to both error_log file and central syslog server?

Have you done any basic network tests to make sure you can connect to your syslog server from the remote machines?

> 
>>> I'm not sure about CustomLog.  That might take a little more work.
>>> Are you trying to send access logs as well?
>>> 
>>>> and Apache tomcat logs
>>> 
>>> First thing that comes to mind here is using Log4j and it's syslog
>>> appender.
>> 
>> +1
>> 
>> I believe there is a syslog plug-in for JULI as well. Not supported
>> here but I believe it works.
>> 
> 
> Currently my log4j.properties looks like this.
> log4j.rootLogger=WARN, stdout

This is directing the root logger to use your ConsoleAppender, not the SyslogAppender.  This means anything that you do not specifically send to the SyslogAppender will be sent to stdout.  Probably not what you want here.

> log4j.appender.stdout=org.apache.log4j.ConsoleAppender
> log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
> log4j.appender.stdout.layout.ConversionPattern=[%d] %5p%6.6r[%t]%x -
> %C.%M(%F:%L) - %m%n
> log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
> log4j.appender.SYSLOG.syslogHost=localhost
> log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
> log4j.appender.SYSLOG.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2}
> %x - %m%n
> log4j.appender.SYSLOG.Facility=LOCAL1
> log4j.appender.SYSLOG.Threshold=debug
> log4j.appender.SYSLOG.FacilityPrinting=true
> 
> But for this also I'm not getting any logs on any server. Do I need to add
> any jar files?

Have you followed the instructions for using Log4j with Tomcat?

  http://tomcat.apache.org/tomcat-7.0-doc/logging.html#Using_Log4j

I’d suggest you get Log4j working to a local file first.  Then add the SyslogAppender and try sending them to your syslog server.  It will make troubleshooting easier if you know that you have Log4j working before you try to setup syslog.

Dan


> -Randeep
> 
>> 
>> - -chris
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>> 
>> iQIcBAEBCAAGBQJTKJ8vAAoJEBzwKT+lPKRY088QAKwvUu4Tm3Mbf+VDMScpHdDe
>> gvJBRmpNgBxEYUh3FC5TUyngrQGiKy24jxeMSvBbcsipZQRPb56n1/G1SoFVewYm
>> +ty6XPngdTLtU8SylLTlB0rKOB/VaZGccmXgiY6wAwRjJwoEMBT/OYXaAfJdCRNV
>> Jnrr2SaDidLxMpdiIC2Wh5lDdPeOtMAadLB3Q7dUGM0SY63a3MXKfHxx+f8VDy/t
>> 57VzGc125U4f9HGyA7JFHcb1cxi6OJjJ8SwwkGtwpGKlPmk8Ut6qcn5zR5D73M+5
>> R6YHPcMN804n7M/8mgbVYCKVmkLnmT/eE5wPfxX9ucJDRSZ1CHLBWr828/2BkA/e
>> JhEsh85tNqOkWbZ8G7VglA3NhHoIPiy2/Isr9Ig+kPR8HxoenQ9yrRxsEs38Y02P
>> DVP53wx0QxuTYpraFkvPROl1riY2U18tglPZBRjaAg84lKq9RsiIhLPLKY9DM/PS
>> GeDfa/MZZUJZLv1iBsO6COIiHdfwZ11S0aXGVB0BsH9vbUR4i1Se9CYxXCq8ph6G
>> Ki5CU1HJ3NaF4gQU8eIP3U+BZRe4gxQkfFQISTQ5++w9qIQZ/yW9LGbwFdRaZ9tG
>> /EyF8cLM6RD/uqLk6Dp3MZN8iyB+LZ3MRwode5ndZd8M41SIwcTuGXDw6cA0hwO0
>> ZajYkHWhEcjx+bIYrUfx
>> =KVxw
>> -----END PGP SIGNATURE-----
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
>> 
> 
> 
> -- 
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> I blog here:
> http://www.randeeppr.me/
> Follow me Here:
> http://twitter.com/Randeeppr
> Poke me here!
> http://www.facebook.com/Randeeppr
> A little Linux Help
> http://www.linuxhelp.in/
> Work profile:
> http://in.linkedin.com/in/randeeppr


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: [Off Topic] Forwarding tomcat logs

Posted by Randeep <ra...@gmail.com>.
Chris,


On Wed, Mar 19, 2014 at 1:01 AM, Christopher Schultz <
chris@christopherschultz.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Dan,
>
> On 3/18/14, 10:40 AM, Daniel Mikusa wrote:
> > On Mar 18, 2014, at 10:13 AM, Randeep <ra...@gmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> I have servers in Amazon Web Services Platform.
> >>
> >> My servers are Centos 5.4
> >>
> >> I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
>

I'm using httpd as front end because I have a few redirection rules set.
Also I was using httpd for handling ssl certificates.

> >>
> >> I'm using elastic load balancer with autoscaling, so, many
> >> instances can spawn and terminate anytime. But once the instance
> >> is terminated tomcat logs of that instance is no more available.
> >>
> >> I was trying to forward httpd
> >
> > ErrorLog supports sending logs to syslog.
> >
> > http://httpd.apache.org/docs/2.2/mod/core.html#errorlog
>
> +1
>
> Syslog is pretty much the answer to the whole question.
>

I have configured my httpd.conf as below
ErrorLog syslog:local7

And syslog.conf as
local7.*                                                 @
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
local1.*                                                 @
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

But I'm not getting any log in error_log or syslog central server.
Is it possible to get logs to both error_log file and central syslog server?


> > I'm not sure about CustomLog.  That might take a little more work.
> > Are you trying to send access logs as well?
> >
> >> and Apache tomcat logs
> >
> > First thing that comes to mind here is using Log4j and it's syslog
> > appender.
>
> +1
>
> I believe there is a syslog plug-in for JULI as well. Not supported
> here but I believe it works.
>

Currently my log4j.properties looks like this.
log4j.rootLogger=WARN, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %5p%6.6r[%t]%x -
%C.%M(%F:%L) - %m%n
log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.syslogHost=localhost
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2}
%x - %m%n
log4j.appender.SYSLOG.Facility=LOCAL1
log4j.appender.SYSLOG.Threshold=debug
log4j.appender.SYSLOG.FacilityPrinting=true

But for this also I'm not getting any logs on any server. Do I need to add
any jar files?
-Randeep

>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJTKJ8vAAoJEBzwKT+lPKRY088QAKwvUu4Tm3Mbf+VDMScpHdDe
> gvJBRmpNgBxEYUh3FC5TUyngrQGiKy24jxeMSvBbcsipZQRPb56n1/G1SoFVewYm
> +ty6XPngdTLtU8SylLTlB0rKOB/VaZGccmXgiY6wAwRjJwoEMBT/OYXaAfJdCRNV
> Jnrr2SaDidLxMpdiIC2Wh5lDdPeOtMAadLB3Q7dUGM0SY63a3MXKfHxx+f8VDy/t
> 57VzGc125U4f9HGyA7JFHcb1cxi6OJjJ8SwwkGtwpGKlPmk8Ut6qcn5zR5D73M+5
> R6YHPcMN804n7M/8mgbVYCKVmkLnmT/eE5wPfxX9ucJDRSZ1CHLBWr828/2BkA/e
> JhEsh85tNqOkWbZ8G7VglA3NhHoIPiy2/Isr9Ig+kPR8HxoenQ9yrRxsEs38Y02P
> DVP53wx0QxuTYpraFkvPROl1riY2U18tglPZBRjaAg84lKq9RsiIhLPLKY9DM/PS
> GeDfa/MZZUJZLv1iBsO6COIiHdfwZ11S0aXGVB0BsH9vbUR4i1Se9CYxXCq8ph6G
> Ki5CU1HJ3NaF4gQU8eIP3U+BZRe4gxQkfFQISTQ5++w9qIQZ/yW9LGbwFdRaZ9tG
> /EyF8cLM6RD/uqLk6Dp3MZN8iyB+LZ3MRwode5ndZd8M41SIwcTuGXDw6cA0hwO0
> ZajYkHWhEcjx+bIYrUfx
> =KVxw
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>


-- 
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
I blog here:
http://www.randeeppr.me/
Follow me Here:
http://twitter.com/Randeeppr
Poke me here!
http://www.facebook.com/Randeeppr
A little Linux Help
http://www.linuxhelp.in/
Work profile:
http://in.linkedin.com/in/randeeppr

Re: [Off Topic] Forwarding tomcat logs

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Dan,

On 3/18/14, 10:40 AM, Daniel Mikusa wrote:
> On Mar 18, 2014, at 10:13 AM, Randeep <ra...@gmail.com>
> wrote:
> 
>> Hi,
>> 
>> I have servers in Amazon Web Services Platform.
>> 
>> My servers are Centos 5.4
>> 
>> I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
>> 
>> I'm using elastic load balancer with autoscaling, so, many
>> instances can spawn and terminate anytime. But once the instance
>> is terminated tomcat logs of that instance is no more available.
>> 
>> I was trying to forward httpd
> 
> ErrorLog supports sending logs to syslog.
> 
> http://httpd.apache.org/docs/2.2/mod/core.html#errorlog

+1

Syslog is pretty much the answer to the whole question.

> I’m not sure about CustomLog.  That might take a little more work.
> Are you trying to send access logs as well?
> 
>> and Apache tomcat logs
> 
> First thing that comes to mind here is using Log4j and it’s syslog
> appender.

+1

I believe there is a syslog plug-in for JULI as well. Not supported
here but I believe it works.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTKJ8vAAoJEBzwKT+lPKRY088QAKwvUu4Tm3Mbf+VDMScpHdDe
gvJBRmpNgBxEYUh3FC5TUyngrQGiKy24jxeMSvBbcsipZQRPb56n1/G1SoFVewYm
+ty6XPngdTLtU8SylLTlB0rKOB/VaZGccmXgiY6wAwRjJwoEMBT/OYXaAfJdCRNV
Jnrr2SaDidLxMpdiIC2Wh5lDdPeOtMAadLB3Q7dUGM0SY63a3MXKfHxx+f8VDy/t
57VzGc125U4f9HGyA7JFHcb1cxi6OJjJ8SwwkGtwpGKlPmk8Ut6qcn5zR5D73M+5
R6YHPcMN804n7M/8mgbVYCKVmkLnmT/eE5wPfxX9ucJDRSZ1CHLBWr828/2BkA/e
JhEsh85tNqOkWbZ8G7VglA3NhHoIPiy2/Isr9Ig+kPR8HxoenQ9yrRxsEs38Y02P
DVP53wx0QxuTYpraFkvPROl1riY2U18tglPZBRjaAg84lKq9RsiIhLPLKY9DM/PS
GeDfa/MZZUJZLv1iBsO6COIiHdfwZ11S0aXGVB0BsH9vbUR4i1Se9CYxXCq8ph6G
Ki5CU1HJ3NaF4gQU8eIP3U+BZRe4gxQkfFQISTQ5++w9qIQZ/yW9LGbwFdRaZ9tG
/EyF8cLM6RD/uqLk6Dp3MZN8iyB+LZ3MRwode5ndZd8M41SIwcTuGXDw6cA0hwO0
ZajYkHWhEcjx+bIYrUfx
=KVxw
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: [Off Topic] Forwarding tomcat logs

Posted by Daniel Mikusa <dm...@gopivotal.com>.
On Mar 18, 2014, at 10:13 AM, Randeep <ra...@gmail.com> wrote:

> Hi,
> 
> I have servers in Amazon Web Services Platform.
> 
> My servers are Centos 5.4
> 
> I have httpd-2.2+mod_jk+tomcat-6.0.37 stack on them
> 
> I'm using elastic load balancer with autoscaling, so, many instances can
> spawn and terminate anytime. But once the instance is terminated tomcat
> logs of that instance is no more available.
> 
> I was trying to forward httpd

ErrorLog supports sending logs to syslog.  

  http://httpd.apache.org/docs/2.2/mod/core.html#errorlog

I’m not sure about CustomLog.  That might take a little more work.  Are you trying to send access logs as well?

> and Apache tomcat logs

First thing that comes to mind here is using Log4j and it’s syslog appender.

You can setup Tomcat to use Log4j.  Instructions here.

  http://tomcat.apache.org/tomcat-7.0-doc/logging.html#Using_Log4j

Then configure an appender to use SyslogAppender.

  http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html

That should get you anything that would normally be written to a Tomcat log (catalina.yyyy-mm-dd.log or localhost.yyyy-mm-dd.log).  It wouldn’t get you access logs or anything your apps are writing to System.out / System.err.  Again, how much are you trying to send to your syslog server?

Dan

> to a central logging
> server (splunk/graylog2). But was not able to.
> 
> http://stackoverflow.com/questions/22293195/forwarding-logs-to-splunk-gralog-from-syslog-ng
> 
> Has anyone done this? What is the best way to do it?
> 
> Kindly advise.
> 
> -- Randeep


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org