You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@camel.apache.org by michaelv <mi...@vanorman.net> on 2009/08/06 17:09:20 UTC

JMS dropping messages

We have an application deployed in Weblogic and I have written a camel client
(using 2.0-M3) that publishes JMS messages to a queue read by that
application and then monitors a couple of queues for different types of
output. So here are my issues:

1. One of the queues, at least, is showing a discrepancy between the number
of messages posted and the number of messages my camel client consumes -- my
client is short. In one common example, a particular set of input messages
generates a total of 94 messages on one of the output queues -- as indicated
by weblogic and my application logging. My camel client, however, only
appears to consume 93 of them, with no errors reported anywhere and no
indication in the logs of any other messages that never reached my Producer.
So my question is, where did it go, or how can I trace it? I've simplified
my route to only a single "to" that simply posts to a bean processor that
does nothing but log the receipt of the message.

2. In an attempt to simplify further, I explicitly set the number of
consumers on the queue to 1 (though I believed that to be the default).
However, I still see anywhere from 1 to as many as 7 consumers connected to
the queue (no, there are no other clients other than this one, the count
drops to 0 immediately upon termination of this client). So why doesn't
camel respect the consumer settings?

Here is the portion of my configuration that applies to this particular JMS
queue:

<camel:route id="errorFlow">
<camel:from
uri="weblogicsp:queue:ecis.helix.service.errorQ?concurrentConsumers=1?msMessageType=Text"
/>
<camel:to uri="bean:tracker?method=receive" />
</camel:route>

NOTE: This client connects to two separate JMS servers, and one uses the
"jms" prefix, so I created a second "weblogicsp" to connect to the other one
-- hence the non-standard prefix. There is another similar configuration
using "jms" that points to that other server, and there is a route that
connects to a queue on that server as well, in case that matters. Both
route replies to the same "bean:tracker?method=receive" endpoint.

--
View this message in context: http://www.nabble.com/JMS-dropping-messages-tp24848390p24848390.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JMS dropping messages

Posted by Claus Ibsen <cl...@gmail.com>.

On Fri, Aug 7, 2009 at 7:15 PM, michaelv<mi...@vanorman.net> wrote:
>
> I found it.  So it turns out it's a bit of a false alarm.  There was another
> process started by our app I wasn't aware of that didn't occasional polling
> on that queue and removed data.  When I wrote the plain Java code I wasn't
> using polling, so I received all the messages.  When using camel / spring it
> uses polling and so it left a brief window for the other process to
> occasionally grab a message.  I was able to disable that other process and
> resolve the issue.
>

Glad you found the culprit. Something like that happens once in a while.


> Thanks!
>
>
> michaelv wrote:
>>
>> Thanks for the tips.  I had already tried a plain JMS client and it worked
>> fine.  I create my own JMS connection / session / listener and then just
>> forwarded the messages to a "direct" endpoint and it received all the
>> messages.
>>
>> I turned on TRACE (thanks for that) and checked the onMessage calls.  As
>> it turns out, onMessage is being called too few times for that queue.  So
>> for that dataset, it should be called 94 times for that queue, but it
>> varies from about 88 to 93 (it's not the same every time).  So it would
>> appear as if the messages aren't even being delivered to the
>> EndpontMessageListener.
>>
>> Changing maxMessagesPerTask had now affect.  I had tried 1 before just to
>> see if that made a difference, but setting it to -1 didn't help either.
>>
>> Any other suggestions?  I'm a bit stumped with this one.  Does the fault
>> likely lie with Spring, or is there something else I may be
>> misconfiguring?
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> Hi
>>>
>>> Strange as we dont get this kind of reportings before.
>>>
>>> The JMS component have TRACE logging (verbose) but you can use it as
>>> it will log at onMessage START when the onMessage method is invoked by
>>> the spring JMS listener. This is the very first code that is executed
>>> in Camel. And maybe that can help you to count to 100.
>>>
>>> For compariion you could build a plain JMS listener and try to see if
>>> you can use it to get all 100 msg as well?
>>>
>>> And you should set this option as well:
>>> maxMessagesPerTask=-1
>>>
>>> I cannot remember if that was fixed in 2.0m3 or later.
>>>
>>>
>>>
>>> On Thu, Aug 6, 2009 at 5:09 PM, michaelv<mi...@vanorman.net> wrote:
>>>>
>>>> We have an application deployed in Weblogic and I have written a camel
>>>> client
>>>> (using 2.0-M3) that publishes JMS messages to a queue read by that
>>>> application and then monitors a couple of queues for different types of
>>>> output.  So here are my issues:
>>>>
>>>> 1.  One of the queues, at least, is showing a discrepancy between the
>>>> number
>>>> of messages posted and the number of messages my camel client consumes
>>>> -- my
>>>> client is short.  In one common example, a particular set of input
>>>> messages
>>>> generates a total of 94 messages on one of the output queues -- as
>>>> indicated
>>>> by weblogic and my application logging.  My camel client, however, only
>>>> appears to consume 93 of them, with no errors reported anywhere and no
>>>> indication in the logs of any other messages that never reached my
>>>> Producer.
>>>> So my question is, where did it go, or how can I trace it?  I've
>>>> simplified
>>>> my route to only a single "to" that simply posts to a bean processor
>>>> that
>>>> does nothing but log the receipt of the message.
>>>>
>>>> 2. In an attempt to simplify further, I explicitly set the number of
>>>> consumers on the queue to 1 (though I believed that to be the default).
>>>> However, I still see anywhere from 1 to as many as 7 consumers connected
>>>> to
>>>> the queue (no, there are no other clients other than this one, the count
>>>> drops to 0 immediately upon termination of this client).  So why doesn't
>>>> camel respect the consumer settings?
>>>>
>>>> Here is the portion of my configuration that applies to this particular
>>>> JMS
>>>> queue:
>>>>
>>>> <camel:route id="errorFlow">
>>>>   <camel:from
>>>> uri="weblogicsp:queue:ecis.helix.service.errorQ?concurrentConsumers=1?msMessageType=Text"
>>>> />
>>>>   <camel:to uri="bean:tracker?method=receive" />
>>>> </camel:route>
>>>>
>>>>
>>>> <bean id="weblogicsp"
>>>> class="org.apache.camel.component.jms.JmsComponent">
>>>>   <property name="connectionFactory" ref="spJmsConnectionFactory" />
>>>>   <property name="destinationResolver" ref="spJmsResolver" />
>>>> </bean>
>>>>
>>>>
>>>> NOTE: This client connects to two separate JMS servers, and one uses the
>>>> "jms" prefix, so I created a second "weblogicsp" to connect to the other
>>>> one
>>>> -- hence the non-standard prefix.  There is another similar
>>>> configuration
>>>> using "jms" that points to that other server, and there is a route that
>>>> connects to a queue on that server as well, in case that matters.  Both
>>>> route replies to the same "bean:tracker?method=receive" endpoint.
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/JMS-dropping-messages-tp24848390p24848390.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/JMS-dropping-messages-tp24848390p24868677.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: JMS dropping messages

Posted by michaelv <mi...@vanorman.net>.

I found it.  So it turns out it's a bit of a false alarm.  There was another
process started by our app I wasn't aware of that didn't occasional polling
on that queue and removed data.  When I wrote the plain Java code I wasn't
using polling, so I received all the messages.  When using camel / spring it
uses polling and so it left a brief window for the other process to
occasionally grab a message.  I was able to disable that other process and
resolve the issue.

Thanks!


michaelv wrote:
> 
> Thanks for the tips.  I had already tried a plain JMS client and it worked
> fine.  I create my own JMS connection / session / listener and then just
> forwarded the messages to a "direct" endpoint and it received all the
> messages.
> 
> I turned on TRACE (thanks for that) and checked the onMessage calls.  As
> it turns out, onMessage is being called too few times for that queue.  So
> for that dataset, it should be called 94 times for that queue, but it
> varies from about 88 to 93 (it's not the same every time).  So it would
> appear as if the messages aren't even being delivered to the
> EndpontMessageListener.
> 
> Changing maxMessagesPerTask had now affect.  I had tried 1 before just to
> see if that made a difference, but setting it to -1 didn't help either.
> 
> Any other suggestions?  I'm a bit stumped with this one.  Does the fault
> likely lie with Spring, or is there something else I may be
> misconfiguring?
> 
> 
> Claus Ibsen-2 wrote:
>> 
>> Hi
>> 
>> Strange as we dont get this kind of reportings before.
>> 
>> The JMS component have TRACE logging (verbose) but you can use it as
>> it will log at onMessage START when the onMessage method is invoked by
>> the spring JMS listener. This is the very first code that is executed
>> in Camel. And maybe that can help you to count to 100.
>> 
>> For compariion you could build a plain JMS listener and try to see if
>> you can use it to get all 100 msg as well?
>> 
>> And you should set this option as well:
>> maxMessagesPerTask=-1
>> 
>> I cannot remember if that was fixed in 2.0m3 or later.
>> 
>> 
>> 
>> On Thu, Aug 6, 2009 at 5:09 PM, michaelv<mi...@vanorman.net> wrote:
>>>
>>> We have an application deployed in Weblogic and I have written a camel
>>> client
>>> (using 2.0-M3) that publishes JMS messages to a queue read by that
>>> application and then monitors a couple of queues for different types of
>>> output.  So here are my issues:
>>>
>>> 1.  One of the queues, at least, is showing a discrepancy between the
>>> number
>>> of messages posted and the number of messages my camel client consumes
>>> -- my
>>> client is short.  In one common example, a particular set of input
>>> messages
>>> generates a total of 94 messages on one of the output queues -- as
>>> indicated
>>> by weblogic and my application logging.  My camel client, however, only
>>> appears to consume 93 of them, with no errors reported anywhere and no
>>> indication in the logs of any other messages that never reached my
>>> Producer.
>>> So my question is, where did it go, or how can I trace it?  I've
>>> simplified
>>> my route to only a single "to" that simply posts to a bean processor
>>> that
>>> does nothing but log the receipt of the message.
>>>
>>> 2. In an attempt to simplify further, I explicitly set the number of
>>> consumers on the queue to 1 (though I believed that to be the default).
>>> However, I still see anywhere from 1 to as many as 7 consumers connected
>>> to
>>> the queue (no, there are no other clients other than this one, the count
>>> drops to 0 immediately upon termination of this client).  So why doesn't
>>> camel respect the consumer settings?
>>>
>>> Here is the portion of my configuration that applies to this particular
>>> JMS
>>> queue:
>>>
>>> <camel:route id="errorFlow">
>>>   <camel:from
>>> uri="weblogicsp:queue:ecis.helix.service.errorQ?concurrentConsumers=1?msMessageType=Text"
>>> />
>>>   <camel:to uri="bean:tracker?method=receive" />
>>> </camel:route>
>>>
>>>
>>> <bean id="weblogicsp"
>>> class="org.apache.camel.component.jms.JmsComponent">
>>>   <property name="connectionFactory" ref="spJmsConnectionFactory" />
>>>   <property name="destinationResolver" ref="spJmsResolver" />
>>> </bean>
>>>
>>>
>>> NOTE: This client connects to two separate JMS servers, and one uses the
>>> "jms" prefix, so I created a second "weblogicsp" to connect to the other
>>> one
>>> -- hence the non-standard prefix.  There is another similar
>>> configuration
>>> using "jms" that points to that other server, and there is a route that
>>> connects to a queue on that server as well, in case that matters.  Both
>>> route replies to the same "bean:tracker?method=receive" endpoint.
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/JMS-dropping-messages-tp24848390p24848390.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>> 
>> 
>> 
>> -- 
>> Claus Ibsen
>> Apache Camel Committer
>> 
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/JMS-dropping-messages-tp24848390p24868677.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JMS dropping messages

Posted by michaelv <mi...@vanorman.net>.

Thanks for the tips.  I had already tried a plain JMS client and it worked
fine.  I create my own JMS connection / session / listener and then just
forwarded the messages to a "direct" endpoint and it received all the
messages.

I turned on TRACE (thanks for that) and checked the onMessage calls.  As it
turns out, onMessage is being called too few times for that queue.  So for
that dataset, it should be called 94 times for that queue, but it varies
from about 88 to 93 (it's not the same every time).  So it would appear as
if the messages aren't even being delivered to the EndpontMessageListener.

Changing maxMessagesPerTask had now affect.  I had tried 1 before just to
see if that made a difference, but setting it to -1 didn't help either.

Any other suggestions?  I'm a bit stumped with this one.  Does the fault
likely lie with Spring, or is there something else I may be misconfiguring?


Claus Ibsen-2 wrote:
> 
> Hi
> 
> Strange as we dont get this kind of reportings before.
> 
> The JMS component have TRACE logging (verbose) but you can use it as
> it will log at onMessage START when the onMessage method is invoked by
> the spring JMS listener. This is the very first code that is executed
> in Camel. And maybe that can help you to count to 100.
> 
> For compariion you could build a plain JMS listener and try to see if
> you can use it to get all 100 msg as well?
> 
> And you should set this option as well:
> maxMessagesPerTask=-1
> 
> I cannot remember if that was fixed in 2.0m3 or later.
> 
> 
> 
> On Thu, Aug 6, 2009 at 5:09 PM, michaelv<mi...@vanorman.net> wrote:
>>
>> We have an application deployed in Weblogic and I have written a camel
>> client
>> (using 2.0-M3) that publishes JMS messages to a queue read by that
>> application and then monitors a couple of queues for different types of
>> output.  So here are my issues:
>>
>> 1.  One of the queues, at least, is showing a discrepancy between the
>> number
>> of messages posted and the number of messages my camel client consumes --
>> my
>> client is short.  In one common example, a particular set of input
>> messages
>> generates a total of 94 messages on one of the output queues -- as
>> indicated
>> by weblogic and my application logging.  My camel client, however, only
>> appears to consume 93 of them, with no errors reported anywhere and no
>> indication in the logs of any other messages that never reached my
>> Producer.
>> So my question is, where did it go, or how can I trace it?  I've
>> simplified
>> my route to only a single "to" that simply posts to a bean processor that
>> does nothing but log the receipt of the message.
>>
>> 2. In an attempt to simplify further, I explicitly set the number of
>> consumers on the queue to 1 (though I believed that to be the default).
>> However, I still see anywhere from 1 to as many as 7 consumers connected
>> to
>> the queue (no, there are no other clients other than this one, the count
>> drops to 0 immediately upon termination of this client).  So why doesn't
>> camel respect the consumer settings?
>>
>> Here is the portion of my configuration that applies to this particular
>> JMS
>> queue:
>>
>> <camel:route id="errorFlow">
>>   <camel:from
>> uri="weblogicsp:queue:ecis.helix.service.errorQ?concurrentConsumers=1?msMessageType=Text"
>> />
>>   <camel:to uri="bean:tracker?method=receive" />
>> </camel:route>
>>
>>
>> <bean id="weblogicsp"
>> class="org.apache.camel.component.jms.JmsComponent">
>>   <property name="connectionFactory" ref="spJmsConnectionFactory" />
>>   <property name="destinationResolver" ref="spJmsResolver" />
>> </bean>
>>
>>
>> NOTE: This client connects to two separate JMS servers, and one uses the
>> "jms" prefix, so I created a second "weblogicsp" to connect to the other
>> one
>> -- hence the non-standard prefix.  There is another similar configuration
>> using "jms" that points to that other server, and there is a route that
>> connects to a queue on that server as well, in case that matters.  Both
>> route replies to the same "bean:tracker?method=receive" endpoint.
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/JMS-dropping-messages-tp24848390p24848390.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> Claus Ibsen
> Apache Camel Committer
> 
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
> 
> 

-- 
View this message in context: http://www.nabble.com/JMS-dropping-messages-tp24848390p24866799.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JMS dropping messages

Posted by Claus Ibsen <cl...@gmail.com>.

Hi

Strange as we dont get this kind of reportings before.

The JMS component have TRACE logging (verbose) but you can use it as
it will log at onMessage START when the onMessage method is invoked by
the spring JMS listener. This is the very first code that is executed
in Camel. And maybe that can help you to count to 100.

For compariion you could build a plain JMS listener and try to see if
you can use it to get all 100 msg as well?

And you should set this option as well:
maxMessagesPerTask=-1

I cannot remember if that was fixed in 2.0m3 or later.



On Thu, Aug 6, 2009 at 5:09 PM, michaelv<mi...@vanorman.net> wrote:
>
> We have an application deployed in Weblogic and I have written a camel client
> (using 2.0-M3) that publishes JMS messages to a queue read by that
> application and then monitors a couple of queues for different types of
> output.  So here are my issues:
>
> 1.  One of the queues, at least, is showing a discrepancy between the number
> of messages posted and the number of messages my camel client consumes -- my
> client is short.  In one common example, a particular set of input messages
> generates a total of 94 messages on one of the output queues -- as indicated
> by weblogic and my application logging.  My camel client, however, only
> appears to consume 93 of them, with no errors reported anywhere and no
> indication in the logs of any other messages that never reached my Producer.
> So my question is, where did it go, or how can I trace it?  I've simplified
> my route to only a single "to" that simply posts to a bean processor that
> does nothing but log the receipt of the message.
>
> 2. In an attempt to simplify further, I explicitly set the number of
> consumers on the queue to 1 (though I believed that to be the default).
> However, I still see anywhere from 1 to as many as 7 consumers connected to
> the queue (no, there are no other clients other than this one, the count
> drops to 0 immediately upon termination of this client).  So why doesn't
> camel respect the consumer settings?
>
> Here is the portion of my configuration that applies to this particular JMS
> queue:
>
> <camel:route id="errorFlow">
>   <camel:from
> uri="weblogicsp:queue:ecis.helix.service.errorQ?concurrentConsumers=1?msMessageType=Text"
> />
>   <camel:to uri="bean:tracker?method=receive" />
> </camel:route>
>
>
> <bean id="weblogicsp" class="org.apache.camel.component.jms.JmsComponent">
>   <property name="connectionFactory" ref="spJmsConnectionFactory" />
>   <property name="destinationResolver" ref="spJmsResolver" />
> </bean>
>
>
> NOTE: This client connects to two separate JMS servers, and one uses the
> "jms" prefix, so I created a second "weblogicsp" to connect to the other one
> -- hence the non-standard prefix.  There is another similar configuration
> using "jms" that points to that other server, and there is a route that
> connects to a queue on that server as well, in case that matters.  Both
> route replies to the same "bean:tracker?method=receive" endpoint.
>
>
>
> --
> View this message in context: http://www.nabble.com/JMS-dropping-messages-tp24848390p24848390.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus