You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Mikael Ståldal <mi...@magine.com> on 2016/01/20 19:20:05 UTC
Re: syslog-ng disconnect causes thread to silently die?
SocketAppender extends AbstractOutputStreamAppender, which requires an
OutputStreamManager, which use an OutputStream. I don't think that an
OutputStream is the proper abstraction for an socket appender using Netty
(or something else else based on NIO).
Probably better to make a new appender (such as NioSocketAppender).
On Wed, Jan 20, 2016 at 6:54 PM, Gary Gregory <ga...@gmail.com>
wrote:
> I do not like this kind of magic because I might have Netty on the CP due
> to a different dependency and should then still be able to not use Netty. I
> would want a setting that says "use netty" or that says "use an instance of
> foo for the manager".
>
> Gary
> On Jan 20, 2016 8:35 AM, "Ralph Goers" <ra...@dslextreme.com> wrote:
>
> > In thinking about this some more we really shouldn’t need a new Appender.
> > Just a different SocketManager implementation. The createSocketManager
> > method just needs to check for the presence of some Netty class.
> >
> > Ralph
> >
> > > On Jan 20, 2016, at 9:27 AM, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > > On Wed, Jan 20, 2016 at 2:31 AM, Ralph Goers <
> ralph.goers@dslextreme.com
> > <ma...@dslextreme.com>>
> > > wrote:
> > >
> > >> My first thought would be to create a more advanced socket appender
> that
> > >> uses netty.
> > >>
> > >
> > > I've used Netty before at work and it is certainly fancy; just for
> > > completeness there is also Apache Mina.
> > >
> > > Gary
> > >
> > >
> > >>
> > >> Ralph
> > >>
> > >>> On Jan 19, 2016, at 5:41 PM, Blake Day <bl...@chewy.com> wrote:
> > >>>
> > >>> I can send more messages, but for a period of time after the socket
> > >> closes, messages are discarded. Is there any way to queue (and block)
> > the
> > >> messages until the socket has been re-established?
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>> On 1/19/16, 7:40 PM, "Ralph Goers" <ra...@dslextreme.com>
> > wrote:
> > >>>>
> > >>>> This shows the SocketAppender threw an exception because the
> > connection
> > >> was closed. The exception was caught by the AppenderControl, which
> > re-threw
> > >> it as an AppenderLoggingException. That exception would have been
> > caught
> > >> by the AsyncAppender, which then ignored it. The SocketAppender then
> > >> re-established the connection, so it isn’t dying. You should still
> be
> > >> able to send more messages.
> > >>>>
> > >>>>> On Jan 19, 2016, at 5:17 PM, Blake Day <bl...@chewy.com> wrote:
> > >>>>>
> > >>>>> I’m having trouble getting additional debug messages, but I did
> get
> > >> the following logs with immediateFail=true on the SocketAppender only.
> > >>>>>
> > >>>>>
> > >>>>> By the way, I’m using an AsyncAppender. Would that cause the
> error
> > >> handling to work differently?
> > >>>>>
> > >>>>>
> > >>>>> 2016-01-19 18:30:11,995 AsyncAppender-ASYNC-SYSLOG ERROR Unable to
> > >> write to stream TCP:localhost:601 for appender SYSLOG
> > >>>>> 2016-01-19 18:30:11,996 AsyncAppender-ASYNC-SYSLOG ERROR An
> exception
> > >> occurred processing Appender SYSLOG
> > >> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> > >> writing to TCP:localhost:601
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:140)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:105)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.callAppenders(AsyncAppender.java:308)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:261)
> > >>>>> Caused by: java.net.SocketException: Broken pipe
> > >>>>> at java.net.SocketOutputStream.socketWrite0(Native Method)
> > >>>>> at
> > >> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> > >>>>> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> > >>>>> at
> > >>
> >
> org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:131)
> > >>>>> ... 8 more
> > >>>>> 2016-01-19 18:30:52,941 Thread-42 DEBUG Connection to localhost:601
> > >> reestablished.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On 1/19/16, 4:59 PM, "Ralph Goers" <ra...@dslextreme.com>
> > >> wrote:
> > >>>>>>
> > >>>>>> Can you enable the debug logging so we can see what is happening
> in
> > >> the Appender?
> > >>>>>>
> > >>>>>> Ralph
> > >>>>>>
> > >>>>>>> On Jan 19, 2016, at 1:32 PM, Blake Day <bl...@chewy.com> wrote:
> > >>>>>>>
> > >>>>>>> Sorry, on further investigation, it appears that the thread does
> > not
> > >> die. But the subsequent log messages are lost from the syslog side. I
> > >> found https://issues.apache.org/jira/browse/LOG4J2-122, but I am not
> > sure
> > >> what to make of it. I’ve also tried playing with immediateFail and
> > >> reconnectionDelayMillis, but nothing seems to have the effect I want.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 1/19/16, 2:36 PM, "Blake Day" <bl...@chewy.com> wrote:
> > >>>>>>>>
> > >>>>>>>> I’m using SocketAppender to log to syslog-ng.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> When logging a message that exceeds 8192 bytes (a syslog-ng
> > >> setting), syslog-ng gives an error and closes the connection. What
> > happens
> > >> immediately after is unclear, but it appears that the java thread just
> > dies
> > >> silently. The thread itself is a quartz scheduler worker thread, but
> I
> > >> thought I’d ask here to see whether you guys think this is a log4j
> > >> related issue. There are no java application logs (console or
> > otherwise)
> > >> to suggest what happened.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Here is the output from syslog-ng’s side:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> 2016-01-19T14:26:34-05:00 ERR syslog-ng[87462]: - Incoming frame
> > >> larger than log_msg_size(); log_msg_size='8192', frame_length='100233'
> > >>>>>>>>
> > >>>>>>>> 2016-01-19T14:26:34-05:00 NOTICE syslog-ng[87462]: - Syslog
> > >> connection closed; fd='34', client='AF_INET(127.0.0.1:10142)',
> > >> local='AF_INET(0.0.0.0:601)'
> > >>>>>>>
> > >>>>>>>
> > ---------------------------------------------------------------------
> > >>>>>>> To unsubscribe, e-mail:
> log4j-user-unsubscribe@logging.apache.org
> > >>>>>>> For additional commands, e-mail:
> > log4j-user-help@logging.apache.org
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > >>>>>> For additional commands, e-mail:
> log4j-user-help@logging.apache.org
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > >>>>> For additional commands, e-mail:
> log4j-user-help@logging.apache.org
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > >>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
> > >>>
> > >>
> >
> ТÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÐÐ¥Fò
> > >> Vç7V'67&–&R RÖÖ –â ÆösF¢×W6W"×Vç7V'67&–&T Æövv–æræ 6†Ræ÷&pФf÷"
> > >> FF—F–öæ  6öÖÖ æG2 RÖÖ –â ÆösF¢×W6W"Ö†VÇ Æövv–æræ 6†Ræ÷&pÐ
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > >> For additional commands, e-mail: log4j-user-help@logging.apache.org
> > >>
> > >>
> > >
> > >
> > > --
> > > E-Mail: garydgregory@gmail.com <ma...@gmail.com> |
> > ggregory@apache.org <ma...@apache.org>
> > > Java Persistence with Hibernate, Second Edition
> > > <http://www.manning.com/bauer3/ <http://www.manning.com/bauer3/>>
> > > JUnit in Action, Second Edition <http://www.manning.com/tahchiev/ <
> > http://www.manning.com/tahchiev/>>
> > > Spring Batch in Action <http://www.manning.com/templier/ <
> > http://www.manning.com/templier/>>
> > > Blog: http://garygregory.wordpress.com <
> > http://garygregory.wordpress.com/>
> > > Home: http://garygregory.com/ <http://garygregory.com/>
> > > Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>
> >
>
--
[image: MagineTV]
*Mikael Ståldal*
Senior software developer
*Magine TV*
mikael.staldal@magine.com
Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com
Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.
Re: syslog-ng disconnect causes thread to silently die?
Posted by Ralph Goers <ra...@dslextreme.com>.
Maybe, but it is probably possible to create an OutputStream that wraps the channel. Although slightly more cumbersome the advantage of not having a second appender is probably worth it. I think Netty also supports UDP too. We are also using an OutputStream for that even though UDP uses Datagrams.
Ralph
> On Jan 20, 2016, at 11:20 AM, Mikael Ståldal <mi...@magine.com> wrote:
>
> SocketAppender extends AbstractOutputStreamAppender, which requires an OutputStreamManager, which use an OutputStream. I don't think that an OutputStream is the proper abstraction for an socket appender using Netty (or something else else based on NIO).
>
> Probably better to make a new appender (such as NioSocketAppender).
>
> On Wed, Jan 20, 2016 at 6:54 PM, Gary Gregory <garydgregory@gmail.com <ma...@gmail.com>> wrote:
> I do not like this kind of magic because I might have Netty on the CP due
> to a different dependency and should then still be able to not use Netty. I
> would want a setting that says "use netty" or that says "use an instance of
> foo for the manager".
>
> Gary
> On Jan 20, 2016 8:35 AM, "Ralph Goers" <ralph.goers@dslextreme.com <ma...@dslextreme.com>> wrote:
>
> > In thinking about this some more we really shouldn’t need a new Appender.
> > Just a different SocketManager implementation. The createSocketManager
> > method just needs to check for the presence of some Netty class.
> >
> > Ralph
> >
> > > On Jan 20, 2016, at 9:27 AM, Gary Gregory <garydgregory@gmail.com <ma...@gmail.com>>
> > wrote:
> > >
> > > On Wed, Jan 20, 2016 at 2:31 AM, Ralph Goers <ralph.goers@dslextreme.com <ma...@dslextreme.com>
> > <mailto:ralph.goers@dslextreme.com <ma...@dslextreme.com>>>
> > > wrote:
> > >
> > >> My first thought would be to create a more advanced socket appender that
> > >> uses netty.
> > >>
> > >
> > > I've used Netty before at work and it is certainly fancy; just for
> > > completeness there is also Apache Mina.
> > >
> > > Gary
> > >
> > >
> > >>
> > >> Ralph
> > >>
> > >>> On Jan 19, 2016, at 5:41 PM, Blake Day <blake@chewy.com <ma...@chewy.com>> wrote:
> > >>>
> > >>> I can send more messages, but for a period of time after the socket
> > >> closes, messages are discarded. Is there any way to queue (and block)
> > the
> > >> messages until the socket has been re-established?
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>> On 1/19/16, 7:40 PM, "Ralph Goers" <ralph.goers@dslextreme.com <ma...@dslextreme.com>>
> > wrote:
> > >>>>
> > >>>> This shows the SocketAppender threw an exception because the
> > connection
> > >> was closed. The exception was caught by the AppenderControl, which
> > re-threw
> > >> it as an AppenderLoggingException. That exception would have been
> > caught
> > >> by the AsyncAppender, which then ignored it. The SocketAppender then
> > >> re-established the connection, so it isn’t dying. You should still be
> > >> able to send more messages.
> > >>>>
> > >>>>> On Jan 19, 2016, at 5:17 PM, Blake Day <blake@chewy.com <ma...@chewy.com>> wrote:
> > >>>>>
> > >>>>> I’m having trouble getting additional debug messages, but I did get
> > >> the following logs with immediateFail=true on the SocketAppender only.
> > >>>>>
> > >>>>>
> > >>>>> By the way, I’m using an AsyncAppender. Would that cause the error
> > >> handling to work differently?
> > >>>>>
> > >>>>>
> > >>>>> 2016-01-19 18:30:11,995 AsyncAppender-ASYNC-SYSLOG ERROR Unable to
> > >> write to stream TCP:localhost:601 for appender SYSLOG
> > >>>>> 2016-01-19 18:30:11,996 AsyncAppender-ASYNC-SYSLOG ERROR An exception
> > >> occurred processing Appender SYSLOG
> > >> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> > >> writing to TCP:localhost:601
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:140)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:105)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.callAppenders(AsyncAppender.java:308)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.appender.AsyncAppender$AsyncThread.run(AsyncAppender.java:261)
> > >>>>> Caused by: java.net.SocketException: Broken pipe
> > >>>>> at java.net.SocketOutputStream.socketWrite0(Native Method)
> > >>>>> at
> > >> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> > >>>>> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> > >>>>> at
> > >>
> > org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:131)
> > >>>>> ... 8 more
> > >>>>> 2016-01-19 18:30:52,941 Thread-42 DEBUG Connection to localhost:601
> > >> reestablished.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> On 1/19/16, 4:59 PM, "Ralph Goers" <ralph.goers@dslextreme.com <ma...@dslextreme.com>>
> > >> wrote:
> > >>>>>>
> > >>>>>> Can you enable the debug logging so we can see what is happening in
> > >> the Appender?
> > >>>>>>
> > >>>>>> Ralph
> > >>>>>>
> > >>>>>>> On Jan 19, 2016, at 1:32 PM, Blake Day <blake@chewy.com <ma...@chewy.com>> wrote:
> > >>>>>>>
> > >>>>>>> Sorry, on further investigation, it appears that the thread does
> > not
> > >> die. But the subsequent log messages are lost from the syslog side. I
> > >> found https://issues.apache.org/jira/browse/LOG4J2-122 <https://issues.apache.org/jira/browse/LOG4J2-122>, but I am not
> > sure
> > >> what to make of it. I’ve also tried playing with immediateFail and
> > >> reconnectionDelayMillis, but nothing seems to have the effect I want.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 1/19/16, 2:36 PM, "Blake Day" <blake@chewy.com <ma...@chewy.com>> wrote:
> > >>>>>>>>
> > >>>>>>>> I’m using SocketAppender to log to syslog-ng.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> When logging a message that exceeds 8192 bytes (a syslog-ng
> > >> setting), syslog-ng gives an error and closes the connection. What
> > happens
> > >> immediately after is unclear, but it appears that the java thread just
> > dies
> > >> silently. The thread itself is a quartz scheduler worker thread, but I
> > >> thought I’d ask here to see whether you guys think this is a log4j
> > >> related issue. There are no java application logs (console or
> > otherwise)
> > >> to suggest what happened.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Here is the output from syslog-ng’s side:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> 2016-01-19T14:26:34-05:00 ERR syslog-ng[87462]: - Incoming frame
> > >> larger than log_msg_size(); log_msg_size='8192', frame_length='100233'
> > >>>>>>>>
> > >>>>>>>> 2016-01-19T14:26:34-05:00 NOTICE syslog-ng[87462]: - Syslog
> > >> connection closed; fd='34', client='AF_INET(127.0.0.1:10142)',
> > >> local='AF_INET(0.0.0.0:601 <http://0.0.0.0:601/>)'
> > >>>>>>>
> > >>>>>>>
> > ---------------------------------------------------------------------
> > >>>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org <ma...@logging.apache.org>
> > >>>>>>> For additional commands, e-mail:
> > log4j-user-help@logging.apache.org <ma...@logging.apache.org>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org <ma...@logging.apache.org>
> > >>>>>> For additional commands, e-mail: log4j-user-help@logging.apache.org <ma...@logging.apache.org>
> > >>>>>
> > >>>>> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org <ma...@logging.apache.org>
> > >>>>> For additional commands, e-mail: log4j-user-help@logging.apache.org <ma...@logging.apache.org>
> > >>>>
> > >>>>
> > >>>>
> > >>>> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org <ma...@logging.apache.org>
> > >>>> For additional commands, e-mail: log4j-user-help@logging.apache.org <ma...@logging.apache.org>
> > >>>
> > >>
> > ТÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÐÐ¥Fò
> > >> Vç7V'67&–&R RÖÖ –â ÆösF¢×W6W"×Vç7V'67&–&T Æövv–æræ 6†Ræ÷&pФf÷"
> > >> FF—F–öæ  6öÖÖ æG2 RÖÖ –â ÆösF¢×W6W"Ö†VÇ Æövv–æræ 6†Ræ÷&pÐ
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org <ma...@logging.apache.org>
> > >> For additional commands, e-mail: log4j-user-help@logging.apache.org <ma...@logging.apache.org>
> > >>
> > >>
> > >
> > >
> > > --
> > > E-Mail: garydgregory@gmail.com <ma...@gmail.com> <mailto:garydgregory@gmail.com <ma...@gmail.com>> |
> > ggregory@apache.org <ma...@apache.org> <mailto:ggregory@apache.org <ma...@apache.org>>
> > > Java Persistence with Hibernate, Second Edition
> > > <http://www.manning.com/bauer3/ <http://www.manning.com/bauer3/> <http://www.manning.com/bauer3/ <http://www.manning.com/bauer3/>>>
> > > JUnit in Action, Second Edition <http://www.manning.com/tahchiev/ <http://www.manning.com/tahchiev/> <
> > http://www.manning.com/tahchiev/ <http://www.manning.com/tahchiev/>>>
> > > Spring Batch in Action <http://www.manning.com/templier/ <http://www.manning.com/templier/> <
> > http://www.manning.com/templier/ <http://www.manning.com/templier/>>>
> > > Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/> <
> > http://garygregory.wordpress.com/ <http://garygregory.wordpress.com/>>
> > > Home: http://garygregory.com/ <http://garygregory.com/> <http://garygregory.com/ <http://garygregory.com/>>
> > > Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory> <http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>>
> >
>
>
>
> --
>
>
> Mikael Ståldal
> Senior software developer
>
> Magine TV
> mikael.staldal@magine.com <ma...@magine.com>
> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com <http://www.magine.com/>
>
> Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message
> (or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case,
> you should destroy this message and kindly notify the sender by reply email.