[jira] [Commented] (CAMEL-16916) camel split with parallel processing true consumes lots of heap space when using camel-ftp

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-16916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421282#comment-17421282 ] 

Claus Ibsen commented on CAMEL-16916:
-------------------------------------

Okay so I can confirm its due to camel-ftp is not singleton producers due they are not thread-safe. There are very few components with this behavior.
Also a problem is that the send processor is creating a producer cache with a capacity of 1, so when you have parallel mode, then there is only 1 producer that can be re-used, so a lot of new producers is created. And there is a potential race condition in the MultiPool that camel-ftp uses (not the others). So its a bit of 3 things going on.

> camel split with parallel processing true consumes lots of heap space when using camel-ftp
> ------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-16916
>                 URL: https://issues.apache.org/jira/browse/CAMEL-16916
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 3.11.1
>         Environment: RHEL Linux 7
> Karaf 3.2.1
> OpenJDK 11
>            Reporter: Tapio Piironen
>            Assignee: Claus Ibsen
>            Priority: Minor
>             Fix For: 3.11.3, 3.13.0
>
>
> Camel split with parallel processing true consumes lots of heap space and eventually (after several huge runs) makes java run out of heap space. Reading 50 000 lines from source and creating 50 000 responses as xml files. Out of heap space happends after 3-4 runs (once a day). Workaround is to make parallelProcessing to false .
> My route:
> <from uri="sftp://server?fileName=list.txt"/>
> <split parallelProcessing="true">
>   <tokenize token="\r\n|\n" xml="false" trim="true"/>
>   <convertBodyTo type="java.lang.String" charset="UTF-8"/>
>   <to uri="bean:collectDataByBody?method=collect"/>
>   <to uri="direct:sendToSftp"/>
>  </split>
> <from uri="direct:sendToSftp"/>
>  <choice>
>  <when>
>  <simple>${body} != null</simple>
>  <marshal>
>  <jaxb contextPath="fi.package.entity"/>
>  </marshal>
>  <to uri="sftp://server/answers?disconnect=true"/>
>  </when>
>  <otherwise></otherwise>
>  </choice>
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)