You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Claus Ibsen (Jira)" <ji...@apache.org> on 2021/09/02 13:00:03 UTC
[jira] [Commented] (CAMEL-16916) camel split with parallel
processing true consumes lots of heap space
[ https://issues.apache.org/jira/browse/CAMEL-16916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17408805#comment-17408805 ]
Claus Ibsen commented on CAMEL-16916:
-------------------------------------
You may want to run splitter in streaming mode, otherwise you still end up reading all data into memory, and if parallel you end up doing this concurrently for files all taking up memory space.
> camel split with parallel processing true consumes lots of heap space
> ---------------------------------------------------------------------
>
> Key: CAMEL-16916
> URL: https://issues.apache.org/jira/browse/CAMEL-16916
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 3.11.1
> Environment: RHEL Linux 7
> Karaf 3.2.1
> OpenJDK 11
> Reporter: Tapio Piironen
> Priority: Major
>
> Camel split with parallel processing true consumes lots of heap space and eventually (after several huge runs) makes java run out of heap space. Reading 50 000 lines from source and creating 50 000 responses as xml files. Out of heap space happends after 3-4 runs (once a day). Workaround is to make parallelProcessing to false .
> My route:
> <from uri="sftp://server?fileName=list.txt"/>
> <split parallelProcessing="true">
> <tokenize token="\r\n|\n" xml="false" trim="true"/>
> <convertBodyTo type="java.lang.String" charset="UTF-8"/>
> <to uri="bean:collectDataByBody?method=collect"/>
> <to uri="direct:sendToSftp"/>
> </split>
> <from uri="direct:sendToSftp"/>
> <choice>
> <when>
> <simple>${body} != null</simple>
> <marshal>
> <jaxb contextPath="fi.package.entity"/>
> </marshal>
> <to uri="sftp://server/answers?disconnect=true"/>
> </when>
> <otherwise></otherwise>
> </choice>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)