Re: Apache Camel, SpringBoot application in PCF results with message failures

[Claus Ibsen <cl...@gmail.com>]

Hi

When you run workloads on cloud platforms you need to consider sizing
- how much cpu cores and memory do you have?
And then build and configure your Camel application accordingly.

For example the split in parallel mode will use a default thread pool
of 10-20 threads an so if you have 30 routes that is 30x that.
Instead of private pool per route, you can create one pool and share
it for all 30 routes.

Also mind that 30 routes that run concurrently may not run faster when
you have limited cpu cores.


On Fri, Jan 28, 2022 at 2:00 PM PraveenKumar KG <pr...@gmail.com> wrote:
>
> Hi,
>
> We have developed Apache Came, SpringBoot based application to read data
> from Oracle table and do some transformation of record and publish to Kafka
> topic. We are using Camel SQL component for DB integration and implemented
> Split & parallel processing pattern to parallelize processing to achieve
> high throughput. Oracle tables are partitioned, so we are creating multiple
> routes, one route per table partition, to speed up the processing. We have
> 30 partitions in table and so created 30 routes.
>
>            from(buildSelectSqlEndpoint(routeId))
>                 .process(new GenericEventProcessor("MessagesReceived"))
>                 .split(body())
>                 .parallelProcessing()
>                 .process(new
> GenericEventProcessor("SplitAndParallelProcessing"))
>                 .process(new InventoryProcessor())
>                 .process(new GenericEventProcessor("ConvertedToAvroFormat"))
>                 .to(buildKafkaProducerEndPoint(routeId));
>
> I tested the application in local laptop with an adequate load and it is
> processing as expected. But when we deployed the application in PCF, we see
> some of the threads are failing. I have enabled the Camel debug log and i
> see below debug log line -
>
> Pipeline - Message exchange has failed: so breaking out of pipeline for
> exchange Due to these thousands of messages are not published to Kafka.
>
> From initial analysis, i figured out Camel is creating one thread for each
> route and based on *maxMessagePerPoll* configuration, it creates number of
> threads equal to *maxMessagePerPoll*. With Split & parallel processing
> enabled, Camel creates additional threads equal to *maxMessagePerPoll* for
> Kafka publish. with this approach, we will have hundreds of threads created
> and not sure any issue with PCF. Also I tried number of routes with 2, to
> check the message delivery failures, and still see hundreds of failures and
> with only 2 routes, increases total processing time for millions of records.
>
> Could you please let me know, how to use Apache Camel with containers like
> PCF? any additional configurations we need to have in PCF or Camel ?
>
>
>
> Thanks & Regards
> Praveen KG



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2