Camel JPA - Clustered Deployment - Quartz Scheduling

[Patrick Bray <pa...@gmail.com>]

Hi All, 

I have a requirement to generate JMS events based on updates to JPA entities
in a database by a separate application.

I currently have a proof of concept camel route up and running using the JPA
consumer to poll the database and query for unprocessed records. 

The issue we have however is that we are deploying our Apache Camel
application into a cluster of 8 servers and need to ensure that each record
is only processed once.

As such I was hoping that I could utilise a clustered quartz scheduler with
JDBC job store to ensure that the polling only occurs on a single node
within the cluster whilst still allowing for High Availability via failover
etc.

Can you please advise if it is possible to either: 

a) Configure the polling behavior of the JPA component to be triggered from
a clustered quartz job?
b) Trigger the JPA consumer query from the camel-quartz component using a
clustered scheduler?

Any assistance you could provide would be much appreciated. 

Thanks,
Patrick Bray



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel JPA - Clustered Deployment - Quartz Scheduling

[dkum003 <DH...@amadeus.com>]

Hi,

I feared this.

Do you by any chance have working example which uses
CronScheduledRoutePolicy and supports clustering ?

This would be very helpful.

Regards,
Manish 



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292p5736811.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel JPA - Clustered Deployment - Quartz Scheduling

[gquintana <ge...@gmail.com>]

I find it strange to have N rows in Quartz tables for N cluster nodes. You
should have one row per route x action on route. I imagine that if you
schedule the same action for the same route on 2 different cluster nodes,
sometimes one cluster node will try to execute the Job scheduler by the
other node: as Camel is storing ScheduledJobState in memory, then it's null
it this case. I wonder why Camel is not storing this ScheduledJobState among
JobParams (it's only a route + action), they would be persisted and shared
by all nodes.



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292p5736660.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel JPA - Clustered Deployment - Quartz Scheduling

[dkum003 <DH...@amadeus.com>]

Hi Patrick,

Yes i do have.

I am trying to achieve the same. I did the following changes

1. Quartz.properties in my resource bundle with
org.quartz.jobStore.isClustered = true

2. Made DB entries.

3. This is my route builder - 
QuartzComponent component = new QuartzComponent();
component.setPropertiesFile("quartz.properties");
getContext().addComponent("quartz", component);

We are using CronScheduledRoutePolicy to start the routes.

Everything works fine. When the routes are deployed in a clustered
environment of 2 servers, 2 entries are made in the quartz tables. The route
starts alternatively, but at times i get this error 

Job threw an unhandled exception. [See nested exception:
java.lang.NullPointerException]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: java.lang.NullPointerException
        at
org.apache.camel.routepolicy.quartz.ScheduledJob.execute(ScheduledJob.java:42)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
        ... 1 more

This does not start the job in both the servers




--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292p5736642.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel JPA - Clustered Deployment - Quartz Scheduling

[Patrick Bray <pa...@gmail.com>]

Hi DJ,

Sorry for the delay in responding I have been away on holiday. Yes we
managed to solve this by using the camel-quartz component and a clustered
jdbc job store. Let me know if you have any specific questions on
implementing this.

Thanks,
Patrick Bray



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292p5736486.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel JPA - Clustered Deployment - Quartz Scheduling

[dkum003 <DH...@amadeus.com>]

Hi Patrick,

Did you find a solution to this ?

I have a similar requirement where i need to trigger a daily cron schedule
on a cluster environment.
So basically, only one of the instance should wake up on cron time and
sleep.

Regards,
DJ



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-JPA-Clustered-Deployment-Quartz-Scheduling-tp5733292p5735708.html
Sent from the Camel - Users mailing list archive at Nabble.com.