You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Jan Bednar (Jira)" <ji...@apache.org> on 2020/01/07 20:55:00 UTC
[jira] [Resolved] (CAMEL-14369) Camel Syslog Consumer doesn't
handle syslog message from netcat/python client
[ https://issues.apache.org/jira/browse/CAMEL-14369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Bednar resolved CAMEL-14369.
--------------------------------
Resolution: Not A Bug
Your netcat test is invalid - You are sending some random string, but you need to follow syslog message format.
Camel Syslog Dataformat can handle either RFC-3164 or RFC-5424 compliant messages (this is documented), but python handler does not follow none of these. _logging.handlers.SysLogHandler_ uses timestamp format specified in RFC-5424, but excludes version, which is in RFC-5424 required. Because missing version field, Camel assumes it is RFC-3164 message and fails to decode timestamp.
In python, you can use some RFC compliant logger - eg [https://github.com/jobec/rfc5424-logging-handler]
If you need to stick with _logging.handlers.SysLogHandler_, the [camel-grok|https://camel.apache.org/components/latest/grok-dataformat.html] should work for you.
{code:java}
.unmarshal().grok("%{SYSLOGLINE}"){code}
> Camel Syslog Consumer doesn't handle syslog message from netcat/python client
> -----------------------------------------------------------------------------
>
> Key: CAMEL-14369
> URL: https://issues.apache.org/jira/browse/CAMEL-14369
> Project: Camel
> Issue Type: Bug
> Components: camel-syslog
> Affects Versions: 3.0.0
> Reporter: imran raza khan
> Priority: Major
>
> I developed syslog server in camel like below
>
> {code:java}
> from("netty:udp://127.0.0.1:514?sync=false&allowDefaultCodec=false")
> .routeId("sysloglistener").routeGroup("sysloglistener-group")
> .unmarshal().syslog()
> .to("log:DEBUG?showBody=true&showHeaders=true");
> {code}
> I tested with simple netcat command and it throws error
>
> {code:java}
> echo "Test syslog message from Netcat" | nc -w1 -u 127.0.0.1 514
> {code}
> **Error Trace:**
>
> {code:java}
> 2020-01-06 23:27:08.756 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Step[ID-imrans-MBP-1578349622523-0-5,DefaultErrorHandler[Unmarshal[org.apache.camel.component.syslog.SyslogDataFormat@270d43a7]]]
> 2020-01-06 23:27:08.757 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: CamelInternalProcessor - UnitOfWork - afterProcess - DefaultErrorHandler[Unmarshal[org.apache.camel.component.syslog.SyslogDataFormat@270d43a7]] - ID-imrans-MBP-1578349622523-0-5
> 2020-01-06 23:27:08.757 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Step[ID-imrans-MBP-1578349622523-0-5,DefaultErrorHandler[Unmarshal[org.apache.camel.component.syslog.SyslogDataFormat@270d43a7]]]
> 2020-01-06 23:27:08.757 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Callback[org.apache.camel.processor.CamelInternalProcessor$$Lambda$1103/433074890@3c888d6f]
> 2020-01-06 23:27:08.761 ERROR 2772 --- [ntExecutorGroup] o.a.c.p.e.DefaultErrorHandler : Failed delivery for (MessageId: ID-imrans-MBP-1578349622523-0-6 on ExchangeId: ID-imrans-MBP-1578349622523-0-5). Exhausted after delivery attempt: 1 caught: java.nio.BufferUnderflowException
>
> Message History
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId ProcessorId Processor Elapsed (ms)
> [sysloglistener ] [sysloglistener ] [from[netty://udp://127.0.0.1:514?allowDefaultCodec=false&sync=false] ] [ 2]
> [sysloglistener ] [unmarshal1 ] [unmarshal[org.apache.camel.model.dataformat.SyslogDataFormat@7ec1effa] ] [ 0]
>
> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> java.nio.BufferUnderflowException: null
> at java.nio.Buffer.nextGetIndex(Buffer.java:500) ~[na:1.8.0_121]
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:135) ~[na:1.8.0_121]
> at org.apache.camel.component.syslog.SyslogConverter.parseMessage(SyslogConverter.java:147) ~[camel-syslog-3.0.0.jar:3.0.0]
> at org.apache.camel.component.syslog.SyslogDataFormat.unmarshal(SyslogDataFormat.java:47) ~[camel-syslog-3.0.0.jar:3.0.0]
> at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:62) ~[camel-support-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryState.run(RedeliveryErrorHandler.java:476) ~[camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:87) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:228) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.component.netty.handlers.ServerChannelHandler.processAsynchronously(ServerChannelHandler.java:141) [camel-netty-3.0.0.jar:3.0.0]
> at org.apache.camel.component.netty.handlers.ServerChannelHandler.channelRead0(ServerChannelHandler.java:112) [camel-netty-3.0.0.jar:3.0.0]
> at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
>
> 2020-01-06 23:27:08.762 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Callback[org.apache.camel.processor.Pipeline$$Lambda$1105/1936839498@779c7a4d]
> 2020-01-06 23:27:08.762 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Step[ID-imrans-MBP-1578349622523-0-5,Pipeline[[Channel[Unmarshal[org.apache.camel.component.syslog.SyslogDataFormat@270d43a7]], Channel[sendTo(log://DEBUG?showBody=true&showHeaders=true)]]]]
> 2020-01-06 23:27:08.762 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Callback[org.apache.camel.processor.CamelInternalProcessor$$Lambda$1103/433074890@730c9c78]
> 2020-01-06 23:27:08.762 DEBUG 2772 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Callback[org.apache.camel.component.netty.handlers.ServerChannelHandler$1@789d178]
> {code}
> I tried with python client
> {code:java}
> $ cat syslog-client1.py
> import logging
> import logging.handlers
>
> my_logger = logging.getLogger('MyLogger')
> my_logger.setLevel(logging.DEBUG)
>
> handler = logging.handlers.SysLogHandler(address = ('127.0.0.1',514))
>
> my_logger.addHandler(handler)
>
> my_logger.debug('this is debug')
> my_logger.critical('this is critical')
> {code}
> this time error is different `*+_Invalid syslog message, missing a mandatory space after month_+*`
>
> {code:java}
> 2020-01-06 23:32:03.846 ERROR 2796 --- [ntExecutorGroup] o.a.c.component.syslog.SyslogConverter : Invalid syslog message, missing a mandatory space after month
> 2020-01-06 23:32:03.846 DEBUG 2796 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Step[ID-imrans-MBP-1578349917797-0-7,DefaultErrorHandler[Unmarshal[org.apache.camel.component.syslog.SyslogDataFormat@1da4b3f9]]]
> 2020-01-06 23:32:03.846 DEBUG 2796 --- [ntExecutorGroup] o.a.c.i.engine.DefaultReactiveExecutor : Queuing reactive work: Callback[org.apache.camel.processor.CamelInternalProcessor$$Lambda$1103/1805110340@57ed6495]
> 2020-01-06 23:32:03.847 ERROR 2796 --- [ntExecutorGroup] o.a.c.p.e.DefaultErrorHandler : Failed delivery for (MessageId: ID-imrans-MBP-1578349917797-0-8 on ExchangeId: ID-imrans-MBP-1578349917797-0-7). Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException
>
> Message History
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId ProcessorId Processor Elapsed (ms)
> [sysloglistener ] [sysloglistener ] [from[netty://udp://127.0.0.1:514?allowDefaultCodec=false&sync=false] ] [ 0]
> [sysloglistener ] [unmarshal1 ] [unmarshal[org.apache.camel.model.dataformat.SyslogDataFormat@682d75eb] ] [ 0]
>
> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> java.lang.NullPointerException: null
> at org.apache.camel.component.syslog.SyslogConverter.parseRfc3164Date(SyslogConverter.java:370) ~[camel-syslog-3.0.0.jar:3.0.0]
> at org.apache.camel.component.syslog.SyslogConverter.parseMessage(SyslogConverter.java:185) ~[camel-syslog-3.0.0.jar:3.0.0]
> at org.apache.camel.component.syslog.SyslogDataFormat.unmarshal(SyslogDataFormat.java:47) ~[camel-syslog-3.0.0.jar:3.0.0]
> at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:62) ~[camel-support-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryState.run(RedeliveryErrorHandler.java:476) ~[camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:87) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:228) [camel-base-3.0.0.jar:3.0.0]
> at org.apache.camel.component.netty.handlers.ServerChannelHandler.processAsynchronously(ServerChannelHandler.java:141) [camel-netty-3.0.0.jar:3.0.0]
> at org.apache.camel.component.netty.handlers.ServerChannelHandler.channelRead0(ServerChannelHandler.java:112) [camel-netty-3.0.0.jar:3.0.0]
> at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:56) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:365) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.43.Final.jar:4.1.43.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
> Above server code works if i send syslog message from client developed in Apache Camel, Please find below link of github repo
> [SyslogApplication|https://github.com/imranrazakhan/SyslogApplication]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)