You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Luiz Eduardo Valmont <va...@alis-sol.com.br> on 2019/08/06 04:24:30 UTC
Help Needed: Splitter and Parallel Processing
Hi, friends!
According to instructions found on this link (splitter)
<https://camel.apache.org/splitter.html> and this other link (bean)
<https://camel.apache.org/bean.html>, I wrote this (simple?) route:
*public* *void* configure() *throws* Exception {
> from( "quartz2://instancesProcessor?cron=" + quartz )
> .routeId( "InstancesProcessor" )
> .process( loadUnprocessedInstancesProcessor )
> .split()
> .body()
> .log( LoggingLevel.WARN, "Body type is:
> ${body.getClass().getName()}" )
> .bean( instanceHandler, "handle" )
> .log( LoggingLevel.WARN, "Handled one of:
> ${body.getClass().getName()}" )
> .end()
> .end()
> .end()
> ;
>
And this is the general output:
2019-08-06 04:04:20.281 TRACE fcdc964e5ee8 --- [aultQuartzScheduler-Orthanc
> Overseer_Worker-1] .r.LoadUnprocessedInstancesProcessorImpl : Found 33
> unprocessed instances
> 2019-08-06 04:04:20.373 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:20.507 INFO fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.h.TagInstanceHandler : Tagged instance [...]
> 2019-08-06 04:04:20.518 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.s.InstanceServiceImpl : Saving Instance [...]
> 2019-08-06 04:04:20.559 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> c.a.r.o.o.h.SaveInstanceHandlerDecorator : Saved instance [...]
> 2019-08-06 04:04:20.608 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Handled one of: model.Instance
> 2019-08-06 04:04:20.610 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:20.617 INFO fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.h.TagInstanceHandler : Tagged instance [...]
> 2019-08-06 04:04:20.621 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.s.InstanceServiceImpl : Saving Instance [...]
> 2019-08-06 04:04:20.625 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> c.a.r.o.o.h.SaveInstanceHandlerDecorator : Saved instance [...]
> 2019-08-06 04:04:20.750 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Handled one of: [..]
> 2019-08-06 04:04:20.751 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:30.017 TRACE fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-2]
> .r.LoadUnprocessedInstancesProcessorImpl : Found 33 unprocessed instances
> 2019-08-06 04:04:30.020 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-2] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:40.020 TRACE fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-3]
> .r.LoadUnprocessedInstancesProcessorImpl : Found 33 unprocessed instances
> 2019-08-06 04:04:40.027 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-3] InstancesProcessor
> : Body type is: model.Instance
>
It basically process 2 instances out of 33 available. And that's it. For
the following interations, it merely logs a single Body type. Since it
executes at least once, I reckon that "type-wise" I'm ok, else nothing
would work. I suppose I'm running out of consumers because of some other
route. But it still beats me because, as the logging shows, the
instanceHandler bean does not hang.
Question: what can be wrong/faulty in my routing setup? I wanted to execute
the instanceHandler in parallel, like X at a time. But one step at a time.
Any help or guidance or guess are mostly welcome. Even wild guesses.
Thanks in advance!
Luiz Eduardo Guida Valmont
(21) 2220-6223
(21) 98663-1337
ALIS - Soluções em Engenharia & Sistemas
AW: Help Needed: Splitter and Parallel Processing
Posted by Burkard Stephan <St...@visana.ch>.
Hi
OK, here is my wild guess :-)
Based on your log output I suspect the following:
1. It starts and finds 33 instances to process
2. After processing the second instance, there is a problem
3. Could be that the log output "Handled one of: [..]" instead of "Handled one of: model.Instance" is an indicator for that.
4. Processing is aborted, perhaps the instanceHandler is swallowing exceptions
5. The next trigger starts, it still finds 33 instances because the already processed ones are not removed from the list of unprocessed
6. It tries to process the first, but fails because it is already processed.
7. Processing is aborted
8. Endless loop between #5 and #7
Cheers
Stephan
-----Ursprüngliche Nachricht-----
Von: Luiz Eduardo Valmont <va...@alis-sol.com.br>
Gesendet: Dienstag, 6. August 2019 06:25
An: users@camel.apache.org
Betreff: Help Needed: Splitter and Parallel Processing
Hi, friends!
According to instructions found on this link (splitter) <https://camel.apache.org/splitter.html> and this other link (bean) <https://camel.apache.org/bean.html>, I wrote this (simple?) route:
*public* *void* configure() *throws* Exception {
> from( "quartz2://instancesProcessor?cron=" + quartz )
> .routeId( "InstancesProcessor" )
> .process( loadUnprocessedInstancesProcessor )
> .split()
> .body()
> .log( LoggingLevel.WARN, "Body type is:
> ${body.getClass().getName()}" )
> .bean( instanceHandler, "handle" )
> .log( LoggingLevel.WARN, "Handled one of:
> ${body.getClass().getName()}" )
> .end()
> .end()
> .end()
> ;
>
And this is the general output:
2019-08-06 04:04:20.281 TRACE fcdc964e5ee8 --- [aultQuartzScheduler-Orthanc
> Overseer_Worker-1] .r.LoadUnprocessedInstancesProcessorImpl : Found 33
> unprocessed instances
> 2019-08-06 04:04:20.373 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:20.507 INFO fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.h.TagInstanceHandler : Tagged instance [...]
> 2019-08-06 04:04:20.518 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.s.InstanceServiceImpl : Saving Instance [...]
> 2019-08-06 04:04:20.559 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> c.a.r.o.o.h.SaveInstanceHandlerDecorator : Saved instance [...]
> 2019-08-06 04:04:20.608 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Handled one of: model.Instance
> 2019-08-06 04:04:20.610 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:20.617 INFO fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.h.TagInstanceHandler : Tagged instance [...]
> 2019-08-06 04:04:20.621 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> b.c.a.r.o.o.s.InstanceServiceImpl : Saving Instance [...]
> 2019-08-06 04:04:20.625 DEBUG fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1]
> c.a.r.o.o.h.SaveInstanceHandlerDecorator : Saved instance [...]
> 2019-08-06 04:04:20.750 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Handled one of: [..]
> 2019-08-06 04:04:20.751 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-1] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:30.017 TRACE fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-2]
> .r.LoadUnprocessedInstancesProcessorImpl : Found 33 unprocessed
> instances
> 2019-08-06 04:04:30.020 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-2] InstancesProcessor
> : Body type is: model.Instance
> 2019-08-06 04:04:40.020 TRACE fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-3]
> .r.LoadUnprocessedInstancesProcessorImpl : Found 33 unprocessed
> instances
> 2019-08-06 04:04:40.027 WARN fcdc964e5ee8 ---
> [aultQuartzScheduler-Orthanc Overseer_Worker-3] InstancesProcessor
> : Body type is: model.Instance
>
It basically process 2 instances out of 33 available. And that's it. For the following interations, it merely logs a single Body type. Since it executes at least once, I reckon that "type-wise" I'm ok, else nothing would work. I suppose I'm running out of consumers because of some other route. But it still beats me because, as the logging shows, the instanceHandler bean does not hang.
Question: what can be wrong/faulty in my routing setup? I wanted to execute the instanceHandler in parallel, like X at a time. But one step at a time.
Any help or guidance or guess are mostly welcome. Even wild guesses.
Thanks in advance!
Luiz Eduardo Guida Valmont
(21) 2220-6223
(21) 98663-1337
ALIS - Soluções em Engenharia & Sistemas