You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by garrydias <ga...@gmail.com> on 2013/01/25 19:43:45 UTC
Camel Quartz Bug
Friends,
I think there´s a bug in camel-quartz component.
When I start 500 camel quartz routes I have 500 triggers fired.
When I start 600 camel quartz routes I have 600 triggers fired.
When I start 700 camel quartz routes I have 700 triggers fired.
When I start 800 camel quartz routes I have 800 triggers fired.
When I start 900 camel quartz routes I have 900 triggers fired.
When I start 1000 camel quartz routes I have 975 triggers fired.
When I start 1000 camel quartz routes I have 899 triggers fired.
When I start 1000 camel quartz routes I have 940 triggers fired.
...
So, when I start 1000 camel quartz routes I have [something less than 1000]
triggers fired.
I attached my context.xml and Java Test Case.
Does somebody knows what is happening?
My production app try start over 3000 camel quartz jobs but not all of them
is fired.
Thanx
MassiveCamelQuartzTest.java
<http://camel.465427.n5.nabble.com/file/n5726285/MassiveCamelQuartzTest.java>
destinationTestContext.xml
<http://camel.465427.n5.nabble.com/file/n5726285/destinationTestContext.xml>
log4j.properties
<http://camel.465427.n5.nabble.com/file/n5726285/log4j.properties>
pom.xml <http://camel.465427.n5.nabble.com/file/n5726285/pom.xml>
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Quartz Bug
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jan 28, 2013 at 12:46 AM, garrydias <ga...@gmail.com> wrote:
> Thanx Christian. You are right. Set Exchange.MAXIMUM_ENDPOINT_CACHE_SIZE
> property before camel1.start() is the key to solve my problem.
>
> This is not a bug after all.
>
Ah well in fact we fixed an issue in the upcoming Camel releases
related to this, hitting the cache limit.
This should be fixed in next releases of Camel.
> Thanx again
>
> []s
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726392.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Camel Quartz Bug
Posted by garrydias <ga...@gmail.com>.
Thanx Christian. You are right. Set Exchange.MAXIMUM_ENDPOINT_CACHE_SIZE
property before camel1.start() is the key to solve my problem.
This is not a bug after all.
Thanx again
[]s
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726392.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Quartz Bug
Posted by Christian Müller <ch...@gmail.com>.
Can you increase the max. cache size for Camel endpoints as described at
[1]?
[1]
http://camel.apache.org/how-do-i-configure-the-maximum-endpoint-cache-size-for-camelcontext.html
Best,
Christian
On Sat, Jan 26, 2013 at 12:11 PM, garrydias <ga...@gmail.com> wrote:
> Adding an output of my test case:
> Triggered Jobs: 1200
> Fired Jobs: 200
> camel-quartz-test.txt
> <http://camel.465427.n5.nabble.com/file/n5726337/camel-quartz-test.txt>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726337.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Re: Camel Quartz Bug
Posted by garrydias <ga...@gmail.com>.
Adding an output of my test case:
Triggered Jobs: 1200
Fired Jobs: 200
camel-quartz-test.txt
<http://camel.465427.n5.nabble.com/file/n5726337/camel-quartz-test.txt>
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726337.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Quartz Bug
Posted by garrydias <ga...@gmail.com>.
Thanks Claus.
I redesigned my scheduler project to solve my problem but is a matter of
time until my application start more than 1000 again.
I couldn´t find nothing related to "reasonable" stateful job limit in the
Quartz docs. Using plain quartz this problem does not happens. Using plain
Quartz I could also fire over 5000 jobs facing a slow down at the job
execution(expected behavior of StatefulJob) only.
And change the threadCount to a different than 10 (default defined by
camel-quarz), the problem still reamains. Camel logs the output bellow,
indicating that 100 threads are started but it is unable to fire more than
1000 jobs.
/279 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel
2.9.2 (CamelContext: camel-1) is starting
280 [main] INFO org.apache.camel.management.ManagementStrategyFactory - JMX
enabled. Using ManagedManagementStrategy.
416 [main] INFO
org.apache.camel.management.DefaultManagementLifecycleStrategy -
StatisticsLevel at All so enabling load performance statistics
502 [main] INFO
org.apache.camel.impl.converter.AnnotationTypeConverterLoader - Found 3
packages with 15 @Converter classes to load
522 [main] INFO org.apache.camel.impl.converter.DefaultTypeConverter -
Loaded 170 core type converters (total 170 type converters)
526 [main] INFO
org.apache.camel.impl.converter.AnnotationTypeConverterLoader - Loaded 2
@Converter classes
532 [main] INFO org.apache.camel.impl.converter.DefaultTypeConverter -
Loaded additional 9 type converters (total 179 type converters) in 0.009
seconds
544 [main] INFO org.apache.camel.component.quartz.QuartzComponent - Loading
Quartz properties file from classpath: quartz.properties
592 [main] INFO org.quartz.core.SchedulerSignalerImpl - Initialized
Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
592 [main] INFO org.quartz.core.QuartzScheduler - Quartz Scheduler v.1.8.5
created.
594 [main] INFO org.quartz.simpl.RAMJobStore - RAMJobStore initialized.
594 [main] INFO org.quartz.core.QuartzScheduler - Scheduler meta-data:
Quartz Scheduler (v1.8.5) 'QuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 100 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support
persistence. and is not clustered.
594 [main] INFO org.quartz.impl.StdSchedulerFactory - Quartz scheduler
'QuartzScheduler' initialized from an externally provided properties
instance.
594 [main] INFO org.quartz.impl.StdSchedulerFactory - Quartz scheduler
version: 1.8.5/
The worse case of this fail I see starting 1200 jobs: camel-quartz always
fire only 200 of these jobs. Doesn´t matter the thread pool size .
Thanks again.
Nice weekend
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726336.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Quartz Bug
Posted by Claus Ibsen <cl...@gmail.com>.
On Sat, Jan 26, 2013 at 9:37 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Fri, Jan 25, 2013 at 8:55 PM, garrydias <ga...@gmail.com> wrote:
>> ... a little more info:
>>
>> When I start 999 camel quartz routes I have 999 triggers fired .
>>
>> Is 999 the limit. Why? There´s some extra configuration of something?
>>
>
> Hi
>
> I think you should look at the Quartz Scheduler if there is a
> "reasonable" limit, what a scheduler
> can handle of statelful jobs/triggers.
>
> Also you should possible consider your design, why do you need 1000s+
> Camel routes that trigger almost
> the same quartz cron trigger?
>
Oh and you may want to add more threads to the quartz scheduler so it
has more power
as you trigger the jobs at the same time.
You can configure this in the quartz.properties file for example.
>
>> Thanx
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726288.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Camel Quartz Bug
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 25, 2013 at 8:55 PM, garrydias <ga...@gmail.com> wrote:
> ... a little more info:
>
> When I start 999 camel quartz routes I have 999 triggers fired .
>
> Is 999 the limit. Why? There´s some extra configuration of something?
>
Hi
I think you should look at the Quartz Scheduler if there is a
"reasonable" limit, what a scheduler
can handle of statelful jobs/triggers.
Also you should possible consider your design, why do you need 1000s+
Camel routes that trigger almost
the same quartz cron trigger?
> Thanx
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726288.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Camel Quartz Bug
Posted by garrydias <ga...@gmail.com>.
... a little more info:
When I start 999 camel quartz routes I have 999 triggers fired .
Is 999 the limit. Why? There´s some extra configuration of something?
Thanx
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Bug-tp5726285p5726288.html
Sent from the Camel - Users mailing list archive at Nabble.com.