[jira] [Comment Edited] (CAMEL-15718) Camel lumberjack server component not thread safe

["Zineb BENDHIBA (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221466#comment-17221466 ] 

Zineb BENDHIBA edited comment on CAMEL-15718 at 10/27/20, 2:18 PM:
-------------------------------------------------------------------

thanks [~ezett]. I need to do a test with filebeat. I'm not sure but I feel that filebeat will create one channel for the two messages (so the same session), to send the two window.

Maybe configuring more flush time would help.
In this [integration test|https://github.com/apache/camel/blob/master/components/camel-lumberjack/src/test/java/org/apache/camel/component/lumberjack/LumberjackComponentTest.java] , we have a test with one connection sending 2 window. If we remove [this line of code|https://github.com/apache/camel/blob/a25e9911b5a250116292a8f7bf0ca24df75f4376/components/camel-lumberjack/src/test/java/org/apache/camel/component/lumberjack/LumberjackUtil.java#L88], we will have unwanted acknowledges.



was (Author: zbendhiba):
thanks [~ezett]. I need to do a test with filebeat. I'm not sure but I feel that filebeat will create one channel for the two messages (so the same session), to send the two window.

Maybe configuring more flush time would help.
In this [integration test|https://github.com/apache/camel/blob/master/components/camel-lumberjack/src/test/java/org/apache/camel/component/lumberjack/LumberjackComponentTest.java , we have a test with one connection sending 2 window. If we remove [this line of code|https://github.com/apache/camel/blob/a25e9911b5a250116292a8f7bf0ca24df75f4376/components/camel-lumberjack/src/test/java/org/apache/camel/component/lumberjack/LumberjackUtil.java#L88], we will have unwanted acknowledges.


> Camel lumberjack server component not thread safe
> -------------------------------------------------
>
>                 Key: CAMEL-15718
>                 URL: https://issues.apache.org/jira/browse/CAMEL-15718
>             Project: Camel
>          Issue Type: Bug
>    Affects Versions: 3.5.0, 3.6.0
>         Environment: camel 3.5.0, filebeat 7.9.1, jdk 1.8.0_242, Debian 10, Linux 4.19.0-6-amd64
>            Reporter: Erik Zimmermann
>            Assignee: Zineb BENDHIBA
>            Priority: Major
>         Attachments: filebeat.CAMEL-15718.yml
>
>
> I am new to camel. I use a filebeat source that delivers logfile data over the lumberjack v2 batch protocol. As a receiving server I use the camel lumberjack component to further process the data in a camel pipeline.
> I realized that the LumberjackSessionHandler of camels lumberjack component is not stateless but is being used by camel for all parallel lumberjack connection requests. Thus, new incoming connections with their own batch windows mess up any ongoing process of the already existing window, e.g. window size settings and counting acknowledges.
> Many different combinations of multicast().parallelProcessing().threads(...) with filebeats workers/pipelines showed the same result.
> The states of the stateful LumberjackSessionHandler are mixed up and the handling of parallel windows is broken which results in unwanted/unfinished acknowledges. As a result it hangs up the whole communication process to the filebeat source.
> The LumberjackSessionHandler is not able to handle multiple threads, but camel uses it as it would be able to. 
> How can I tell camel to use separate LumberjackSessionHandlers and processing pipelines for each lumberjack batch request? Or do I misunderstand the concept of how camel uses components?
> Sorry, I wasn't able to find a more specific "camel-lumberjack" component in the list of issue proposals above so I selected camel-core...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)