You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by woggle23 <ma...@yahoo.co.uk> on 2011/06/13 13:08:09 UTC
Multi-threading query
Morning Team
I have a system with 2 routes similar to the following:
HTTP Endpoint -> xmlToSql -> JDBC -> insert/update -> JDBC
POLL-Database -> JDBC -> update/delete -> JDBC
The 1st route receives an xml message via HTTP, then perfoms a query on the
database (to find if an entry relating to the received message already
exists). The outcome of this query is used to generate a 2nd query to insert
the new message, or update the existing message in the database.
The 2nd route performs a timed poll of the database to remove any redundant
entires, which is performed via 2 queries; 1 to read out the entire
contents, the other to update or delete entries based on the outcome of the
initial query.
Due to multi-threading there is potential for route 2 to delete an entry
moments prior to route 1 attempting to update that same entry. How is this
kind of problem tackled in Camel
Many thanks in advance
Matt
--
View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4483959.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Multi-threading query
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Jun 14, 2011 at 1:09 PM, woggle23 <ma...@yahoo.co.uk> wrote:
> I was hoping there would be some form of mutual exclusion lock that I could
> enclose the pertinent elements of my route in (within the dsl statement).
>
You can use a routePolicy which can help dictate when a route is
online. The Camel in Action book covers such use cases in chapter 13.
> Howsoever, the alternative option you mention "detect that the 1st route is
> active". By active do you "not suspended" or "is currently routing a
> message"? If it's the latter, how is this achieved? I know the ServiceStatus
> of a route can be checked via the camel context, but this just provides
> started/stopped/suspended info.
>
The in-flight registry can tell you how many messages a route is
currently processing.
> --
> View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4487483.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Multi-threading query
Posted by woggle23 <ma...@yahoo.co.uk>.
I was hoping there would be some form of mutual exclusion lock that I could
enclose the pertinent elements of my route in (within the dsl statement).
Howsoever, the alternative option you mention "detect that the 1st route is
active". By active do you "not suspended" or "is currently routing a
message"? If it's the latter, how is this achieved? I know the ServiceStatus
of a route can be checked via the camel context, but this just provides
started/stopped/suspended info.
--
View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4487483.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Multi-threading query
Posted by Hadrian Zbarcea <hz...@gmail.com>.
You can also let the update happen and handle the Exception that will
follow. This is an application level problem.
Hadrian
On 06/14/2011 02:27 AM, Claus Ibsen wrote:
> On Mon, Jun 13, 2011 at 1:08 PM, woggle23<ma...@yahoo.co.uk> wrote:
>> Morning Team
>>
>> I have a system with 2 routes similar to the following:
>>
>> HTTP Endpoint -> xmlToSql -> JDBC -> insert/update -> JDBC
>>
>> POLL-Database -> JDBC -> update/delete -> JDBC
>>
>> The 1st route receives an xml message via HTTP, then perfoms a query on the
>> database (to find if an entry relating to the received message already
>> exists). The outcome of this query is used to generate a 2nd query to insert
>> the new message, or update the existing message in the database.
>>
>> The 2nd route performs a timed poll of the database to remove any redundant
>> entires, which is performed via 2 queries; 1 to read out the entire
>> contents, the other to update or delete entries based on the outcome of the
>> initial query.
>>
>> Due to multi-threading there is potential for route 2 to delete an entry
>> moments prior to route 1 attempting to update that same entry. How is this
>> kind of problem tackled in Camel
>>
>
> Well its a JDCB issue. Thats a general problem. There are ways to
> configure isolation levels and whatnot on the JDBC so you can control
> this.
>
> An alternative is in the 2nd route to "detect" that the 1st route is
> active, and then it just skip doing anything this time.
> Or something like that.
>
>> Many thanks in advance
>>
>> Matt
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4483959.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
>
Re: Multi-threading query
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jun 13, 2011 at 1:08 PM, woggle23 <ma...@yahoo.co.uk> wrote:
> Morning Team
>
> I have a system with 2 routes similar to the following:
>
> HTTP Endpoint -> xmlToSql -> JDBC -> insert/update -> JDBC
>
> POLL-Database -> JDBC -> update/delete -> JDBC
>
> The 1st route receives an xml message via HTTP, then perfoms a query on the
> database (to find if an entry relating to the received message already
> exists). The outcome of this query is used to generate a 2nd query to insert
> the new message, or update the existing message in the database.
>
> The 2nd route performs a timed poll of the database to remove any redundant
> entires, which is performed via 2 queries; 1 to read out the entire
> contents, the other to update or delete entries based on the outcome of the
> initial query.
>
> Due to multi-threading there is potential for route 2 to delete an entry
> moments prior to route 1 attempting to update that same entry. How is this
> kind of problem tackled in Camel
>
Well its a JDCB issue. Thats a general problem. There are ways to
configure isolation levels and whatnot on the JDBC so you can control
this.
An alternative is in the 2nd route to "detect" that the 1st route is
active, and then it just skip doing anything this time.
Or something like that.
> Many thanks in advance
>
> Matt
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4483959.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Multi-threading query
Posted by boday <be...@initekconsulting.com>.
Take a look at http://activemq.apache.org/message-groups.html ActiveMQ
Message Groups ...it is designed to provide concurrent processing while
single threading related messages, etc...
Also, you can roll your own locking implementation using
http://camel.apache.org/hazelcast-component.html camel-hazelcast ...
-----
Ben O'Day
IT Consultant -http://consulting-notes.com
--
View this message in context: http://camel.465427.n5.nabble.com/Multi-threading-query-tp4483959p4485861.html
Sent from the Camel - Users mailing list archive at Nabble.com.