MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke <pe...@etone.de>]

Hello,
I observe a very strange behaviour with concurrent tailable consumers and
persistent tail tracking. Perhaps you have an idea:

Scenario:
- A MongoDB Replicaset of 3 servers, running MongoDB 2.4.11
- 2 camel servers (tomcat7 with Sun JDK, Camel 2.13.1)
- On each server, 5 tailable cursor consumer endpoints, listening on 5
different collections with persistent tail tracking enabled
- The persistent id for each consumer endpoint is defined as:
  "<hostname>_<CollectionName>"
- So the collection storing the persistent tail tracking information
("consumedUpdates") looks something like this:



The endpoint configuration looks something like this:

mongodb://mongoDBInstance?collection=<COLLECTION>&database=...&persistentId=<HOST>_<COLLECTION>&persistentTailTracking=true&tailTrackCollection=consumedUpdates&tailTrackIncreasingField=updateId

The problem: Only one document in the "consumedUpdates" collection seems to
be updated, no matter which collection or host is affected. It seems like
this is always the first document in the collection. So all consumers write
into the same document, overwriting information coming from other consumers
:(

The problem does not occur on our test environment (1 camel server, 1
MongoDB, both on the same machine).

Any ideas?

Thanks for your help!

Best regards,
Joerg





--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke <pe...@etone.de>]

Hi,

Claus Ibsen-2 wrote
> Well spotted the bug. Feel free to log a JIRA ticket and provide a patch
> http://camel.apache.org/contributing.html

Thanks.  I submitted a Bug in JIRA.
<https://issues.apache.org/jira/browse/CAMEL-8063>  

Thanks for this great software, by the way (and exspecially @Claus: Thanks
for "Camel in Action", helped me a LOT doing my first steps in camel riding
:))!





--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131p5759327.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB: Persistent tail tracking with concurrent tailable consumers

[Claus Ibsen <cl...@gmail.com>]

Hi

Well spotted the bug. Feel free to log a JIRA ticket and provide a patch
http://camel.apache.org/contributing.html

On Mon, Nov 17, 2014 at 2:11 PM, jpeschke <pe...@etone.de> wrote:
> Hi,
> It's me again.
> I think I found the problem:
>
> It's in the method "initialize" in "MongoDbTailTrackingManager.java":
>
>
>
> If no tail tracking object exists in the database, a new one will be
> inserted (that's okay), but the query "dbCol.findOne()" fetches ANY tail
> tracking object from the database (and not the one we've just inserted).
>
> In my oppinition, this is a bug and should be corrected like this:
>
>
> For now, I work around this by putting every persistent tail tracker in it's
> own collection so I make sure that even a findOne() always gives me the
> right tracker.
>
> Best regards,
> Joerg
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131p5759202.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Re: MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke <pe...@etone.de>]

Hi,
It's me again.
I think I found the problem:

It's in the method "initialize" in "MongoDbTailTrackingManager.java":



If no tail tracking object exists in the database, a new one will be
inserted (that's okay), but the query "dbCol.findOne()" fetches ANY tail
tracking object from the database (and not the one we've just inserted).

In my oppinition, this is a bug and should be corrected like this:


For now, I work around this by putting every persistent tail tracker in it's
own collection so I make sure that even a findOne() always gives me the
right tracker.

Best regards,
Joerg



--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131p5759202.html
Sent from the Camel - Users mailing list archive at Nabble.com.