You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Jawad Ahmad (Jira)" <ji...@apache.org> on 2021/06/08 07:53:00 UTC
[jira] [Resolved] (CAMEL-16696) Enrich and Aggregate Causing
OutOfMemory
[ https://issues.apache.org/jira/browse/CAMEL-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jawad Ahmad resolved CAMEL-16696.
---------------------------------
Resolution: Invalid
> Enrich and Aggregate Causing OutOfMemory
> ----------------------------------------
>
> Key: CAMEL-16696
> URL: https://issues.apache.org/jira/browse/CAMEL-16696
> Project: Camel
> Issue Type: Bug
> Components: came-core
> Affects Versions: 3.10.0
> Reporter: Jawad Ahmad
> Priority: Major
>
> I have a route that receives the bulk amount of data and then processes data individually from 3rd Party API using Enrich Pattern. After Enriching I am then aggregating the responses into a batch of 100 and then sending them to our system. This causes the camel to allocate memory like crazy, stuck into the loop and eventually going out of memory. I tried with Sync routes as well but it gets failed.
> Here is the route configuration :
> {code:java}
> from("direct:sync-likes-from-sf-reviews")
> .id("direct:sync-likes-from-sf-reviews")
> .process(this::convertReviewPayload)
> .split(body())
> .to("direct:enrich-and-fetch-likes")
> .end();
> from("direct:enrich-and-fetch-likes")
> .id("direct-enrich-and-fetch-likes")
> .enrich("direct:fetch-like-by-sf-review", new ReviewLikesAggregation(), true)
> .aggregate(constant(true), new GroupedBodyAggregationStrategy())
> .completionSize(100)
> .to("direct:upsert-collected-likes")
> .end();
> from("direct:upsert-collected-likes")
> .id("direct:upsert-collected-likes")
> .multicast()
> .to("direct:upsert-consumers-batch").to("direct:upsert-likes-batch")
> .end()
> .end();
> from("direct:upsert-consumers-batch?synchronous=false")
> .to("direct:upsert-consumers-batch")
> .process(this.trustpilotLikeConsumersMapper)
> .process(this::setupConsumerHeaders)
> .to("direct:upsert-collections")
> .end();
> from("direct:upsert-likes-batch?synchronous=false")
> .to("direct:upsert-likes-batch")
> .process(this.trustPilotLikesMapper)
> .process(this::setupLikesHeaders)
> .to("direct:upsert-collections")
> .end();
> {code}
> Logs Loop :
> 2021-06-08 12:41:18.568 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.568 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56] o.apache.camel.processor.SendProcessor : >>>> direct://upsert-consumers-batch Exchange[]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)