You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Alistair Young <Al...@uhi.ac.uk> on 2011/09/20 12:18:42 UTC
Random persistent messages lost
Is there anything that might cause a message to go missing? A producer sends a persistent message to a topic but now and then, although no error is returned from the broker, the message just disappears. It's not logged in the data store and it's as if it never reached the broker but that would have produced an error for the producer to catch.
thanks,
Alistair
--
mov eax,1
mov ebx,0
int 80h
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
so a batch of 6 messages came into the broker, 5 disappeared. They were not persisted. One was delivered to the consumer and the rest just simply vanished. tcpdump proves they arrived at the boker but never made it to the consumer.
The activemq logs show the same set of entries for everything that arrives. There doesn't seem to be any error logging.
It seems to sort itself after a restart of the broker which is obviously not a solution but suggests a bug somewhere in either activemq or camel. Can't tell from the logs.
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 27 Sep 2011, at 14:58, Alistair Young wrote:
> I resorted to tcpdump on the broker and the consumer to track the messages and it seems the consumer is dropping messages. Of 16 incoming identical messages, it only acted upon 3. Also, it failed to act on another single message. By that, it didn't pass them to the message handler. tcpdump showed the messages arriving at the consumer. Is there anything I could poke to see why the consumer would discard messages?
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 27 Sep 2011, at 11:40, Alistair Young wrote:
>
>> this is the code that does the sending. I've just watched a batch of three messages being sent, two simply disappeared.
>>
>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>> connection = connectionFactory.createConnection();
>> connection.setClientID(clientId);
>> connection.start();
>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>> destination = session.createTopic(topicName);
>> producer = session.createProducer(destination);
>> producer.setDeliveryMode(messageMode);
>> TextMessage message = session.createTextMessage(messageText);
>> producer.send(message);
>>
>> I tried to create a simple test but the embedded broker never calls isAllowedToConsume
>>
>> Alistair
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 27 Sep 2011, at 11:36, Alistair Young wrote:
>>
>>> the topic consumers don't need to be present when the message is sent as the messages are persistent and they are durable topic subscribers. They are never persisted by activemq. They just disappear. I can't track the messages through the system as isAllowedToConsume is never called for the original message. It's only called when camel reroutes the message and camel never sees the missing messages, as verified by org.apache.camel.Processor instances.
>>>
>>> Alistair
>>>
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 27 Sep 2011, at 11:25, Gary Tully wrote:
>>>
>>>> please provide some code that demonstrates your problem. Are you sure
>>>> that the topic consumers are present when the messages are sent?
>>>> Try and reduce the complexity of your usecase, the bottom line is that
>>>> messages won't just disappear without good reason, there are hundreds
>>>> of test cases that verify this is the case.
>>>>
>>>>
>>>> On 27 September 2011 11:16, Alistair Young <al...@uhi.ac.uk> wrote:
>>>>> not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
>>>>>
>>>>> message -> topicA -> camel -> topicB
>>>>>
>>>>> isAllowedToConsume is never called for topicA
>>>>> isAllowedToConsume is called for topicB with a destination of topic://topicB
>>>>>
>>>>> Alistair
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 26 Sep 2011, at 16:19, Alistair Young wrote:
>>>>>
>>>>>> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>>>>>>
>>>>>> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>>>>>>
>>>>>> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>>>>>>
>>>>>> so on one server the offset and length are wrong as it seems to report too many chars
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>>>>>>
>>>>>>> Is this code shared by multiple threads? does it need synchronization?
>>>>>>>
>>>>>>> What you are experiencing does seem odd, it would be great if you
>>>>>>> could provide a simple junit test case that can reproduce.
>>>>>>>
>>>>>>> Also, peeking that the code of the logging plugin, it should be
>>>>>>> logging sends, that is again odd:
>>>>>>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>>>>>>
>>>>>>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>>>>>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> Alistair
>>>>>>>>
>>>>>>>> try {
>>>>>>>> // Try to reuse a previous connection
>>>>>>>> if (previousMxConnection == null) {
>>>>>>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>>>>>>> connection = connectionFactory.createConnection();
>>>>>>>> connection.setClientID(clientId);
>>>>>>>> connection.start();
>>>>>>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>>>>>>> destination = session.createTopic(topicName);
>>>>>>>> producer = session.createProducer(destination);
>>>>>>>> producer.setDeliveryMode(messageMode);
>>>>>>>>
>>>>>>>> mxConnection = new MatrixConnection();
>>>>>>>> mxConnection.connection = connection;
>>>>>>>> mxConnection.session = session;
>>>>>>>> mxConnection.destination = destination;
>>>>>>>> mxConnection.producer = producer;
>>>>>>>> }
>>>>>>>> else {
>>>>>>>> session = previousMxConnection.session;
>>>>>>>> if (previousMxConnection.producer == null) {
>>>>>>>> producer = session.createProducer(previousMxConnection.destination);
>>>>>>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>>>>> previousMxConnection.producer = producer;
>>>>>>>> }
>>>>>>>> else {
>>>>>>>> producer = previousMxConnection.producer;
>>>>>>>> }
>>>>>>>> mxConnection = previousMxConnection;
>>>>>>>> }
>>>>>>>>
>>>>>>>> TextMessage message = session.createTextMessage(messageText);
>>>>>>>> producer.send(message);
>>>>>>>>
>>>>>>>> return mxConnection;
>>>>>>>> }
>>>>>>>> catch(Exception e) {
>>>>>>>> throw new MatrixClientException(e);
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> mov eax,1
>>>>>>>> mov ebx,0
>>>>>>>> int 80h
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>>>>>>
>>>>>>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>>>>>>
>>>>>>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>>>>>>
>>>>>>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>>>>>>
>>>>>>>>> Alistair
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> mov eax,1
>>>>>>>>> mov ebx,0
>>>>>>>>> int 80h
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>>>>>>
>>>>>>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>>>>>>
>>>>>>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>>>>>>> http://activemq.apache.org/async-sends.html
>>>>>>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>>>>>>
>>>>>>>>>> I'll try this on the broker connection to see if it helps:
>>>>>>>>>>
>>>>>>>>>> jms.useAsyncSend=false
>>>>>>>>>>
>>>>>>>>>> Alistair
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> mov eax,1
>>>>>>>>>> mov ebx,0
>>>>>>>>>> int 80h
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>>>>>>
>>>>>>>>>>> What is the exact broker version used on your side?
>>>>>>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>>>>>>
>>>>>>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>>>>>>
>>>>>>>>>>>> Alistair
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> mov eax,1
>>>>>>>>>>>> mov ebx,0
>>>>>>>>>>>> int 80h
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> http://fusesource.com
>>>>>>> http://blog.garytully.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> http://fusesource.com
>>>> http://blog.garytully.com
>>>
>>
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
I resorted to tcpdump on the broker and the consumer to track the messages and it seems the consumer is dropping messages. Of 16 incoming identical messages, it only acted upon 3. Also, it failed to act on another single message. By that, it didn't pass them to the message handler. tcpdump showed the messages arriving at the consumer. Is there anything I could poke to see why the consumer would discard messages?
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 27 Sep 2011, at 11:40, Alistair Young wrote:
> this is the code that does the sending. I've just watched a batch of three messages being sent, two simply disappeared.
>
> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
> connection = connectionFactory.createConnection();
> connection.setClientID(clientId);
> connection.start();
> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
> destination = session.createTopic(topicName);
> producer = session.createProducer(destination);
> producer.setDeliveryMode(messageMode);
> TextMessage message = session.createTextMessage(messageText);
> producer.send(message);
>
> I tried to create a simple test but the embedded broker never calls isAllowedToConsume
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 27 Sep 2011, at 11:36, Alistair Young wrote:
>
>> the topic consumers don't need to be present when the message is sent as the messages are persistent and they are durable topic subscribers. They are never persisted by activemq. They just disappear. I can't track the messages through the system as isAllowedToConsume is never called for the original message. It's only called when camel reroutes the message and camel never sees the missing messages, as verified by org.apache.camel.Processor instances.
>>
>> Alistair
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 27 Sep 2011, at 11:25, Gary Tully wrote:
>>
>>> please provide some code that demonstrates your problem. Are you sure
>>> that the topic consumers are present when the messages are sent?
>>> Try and reduce the complexity of your usecase, the bottom line is that
>>> messages won't just disappear without good reason, there are hundreds
>>> of test cases that verify this is the case.
>>>
>>>
>>> On 27 September 2011 11:16, Alistair Young <al...@uhi.ac.uk> wrote:
>>>> not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
>>>>
>>>> message -> topicA -> camel -> topicB
>>>>
>>>> isAllowedToConsume is never called for topicA
>>>> isAllowedToConsume is called for topicB with a destination of topic://topicB
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 26 Sep 2011, at 16:19, Alistair Young wrote:
>>>>
>>>>> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>>>>>
>>>>> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>>>>>
>>>>> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>>>>>
>>>>> so on one server the offset and length are wrong as it seems to report too many chars
>>>>>
>>>>> Alistair
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>>>>>
>>>>>> Is this code shared by multiple threads? does it need synchronization?
>>>>>>
>>>>>> What you are experiencing does seem odd, it would be great if you
>>>>>> could provide a simple junit test case that can reproduce.
>>>>>>
>>>>>> Also, peeking that the code of the logging plugin, it should be
>>>>>> logging sends, that is again odd:
>>>>>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>>>>>
>>>>>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>>>>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> Alistair
>>>>>>>
>>>>>>> try {
>>>>>>> // Try to reuse a previous connection
>>>>>>> if (previousMxConnection == null) {
>>>>>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>>>>>> connection = connectionFactory.createConnection();
>>>>>>> connection.setClientID(clientId);
>>>>>>> connection.start();
>>>>>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>>>>>> destination = session.createTopic(topicName);
>>>>>>> producer = session.createProducer(destination);
>>>>>>> producer.setDeliveryMode(messageMode);
>>>>>>>
>>>>>>> mxConnection = new MatrixConnection();
>>>>>>> mxConnection.connection = connection;
>>>>>>> mxConnection.session = session;
>>>>>>> mxConnection.destination = destination;
>>>>>>> mxConnection.producer = producer;
>>>>>>> }
>>>>>>> else {
>>>>>>> session = previousMxConnection.session;
>>>>>>> if (previousMxConnection.producer == null) {
>>>>>>> producer = session.createProducer(previousMxConnection.destination);
>>>>>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>>>> previousMxConnection.producer = producer;
>>>>>>> }
>>>>>>> else {
>>>>>>> producer = previousMxConnection.producer;
>>>>>>> }
>>>>>>> mxConnection = previousMxConnection;
>>>>>>> }
>>>>>>>
>>>>>>> TextMessage message = session.createTextMessage(messageText);
>>>>>>> producer.send(message);
>>>>>>>
>>>>>>> return mxConnection;
>>>>>>> }
>>>>>>> catch(Exception e) {
>>>>>>> throw new MatrixClientException(e);
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> mov eax,1
>>>>>>> mov ebx,0
>>>>>>> int 80h
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>>>>>
>>>>>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>>>>>
>>>>>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>>>>>
>>>>>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>>>>>
>>>>>>>> Alistair
>>>>>>>>
>>>>>>>> --
>>>>>>>> mov eax,1
>>>>>>>> mov ebx,0
>>>>>>>> int 80h
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>>>>>
>>>>>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>>>>>
>>>>>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>>>>>> http://activemq.apache.org/async-sends.html
>>>>>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>>>>>
>>>>>>>>> I'll try this on the broker connection to see if it helps:
>>>>>>>>>
>>>>>>>>> jms.useAsyncSend=false
>>>>>>>>>
>>>>>>>>> Alistair
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> mov eax,1
>>>>>>>>> mov ebx,0
>>>>>>>>> int 80h
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>>>>>
>>>>>>>>>> What is the exact broker version used on your side?
>>>>>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>>>>>
>>>>>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>>>>>
>>>>>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>>>>>
>>>>>>>>>>> Alistair
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> mov eax,1
>>>>>>>>>>> mov ebx,0
>>>>>>>>>>> int 80h
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> http://fusesource.com
>>>>>> http://blog.garytully.com
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> http://fusesource.com
>>> http://blog.garytully.com
>>
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
this is the code that does the sending. I've just watched a batch of three messages being sent, two simply disappeared.
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
connection = connectionFactory.createConnection();
connection.setClientID(clientId);
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
destination = session.createTopic(topicName);
producer = session.createProducer(destination);
producer.setDeliveryMode(messageMode);
TextMessage message = session.createTextMessage(messageText);
producer.send(message);
I tried to create a simple test but the embedded broker never calls isAllowedToConsume
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 27 Sep 2011, at 11:36, Alistair Young wrote:
> the topic consumers don't need to be present when the message is sent as the messages are persistent and they are durable topic subscribers. They are never persisted by activemq. They just disappear. I can't track the messages through the system as isAllowedToConsume is never called for the original message. It's only called when camel reroutes the message and camel never sees the missing messages, as verified by org.apache.camel.Processor instances.
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 27 Sep 2011, at 11:25, Gary Tully wrote:
>
>> please provide some code that demonstrates your problem. Are you sure
>> that the topic consumers are present when the messages are sent?
>> Try and reduce the complexity of your usecase, the bottom line is that
>> messages won't just disappear without good reason, there are hundreds
>> of test cases that verify this is the case.
>>
>>
>> On 27 September 2011 11:16, Alistair Young <al...@uhi.ac.uk> wrote:
>>> not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
>>>
>>> message -> topicA -> camel -> topicB
>>>
>>> isAllowedToConsume is never called for topicA
>>> isAllowedToConsume is called for topicB with a destination of topic://topicB
>>>
>>> Alistair
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 26 Sep 2011, at 16:19, Alistair Young wrote:
>>>
>>>> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>>>>
>>>> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>>>>
>>>> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>>>>
>>>> so on one server the offset and length are wrong as it seems to report too many chars
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>>>>
>>>>> Is this code shared by multiple threads? does it need synchronization?
>>>>>
>>>>> What you are experiencing does seem odd, it would be great if you
>>>>> could provide a simple junit test case that can reproduce.
>>>>>
>>>>> Also, peeking that the code of the logging plugin, it should be
>>>>> logging sends, that is again odd:
>>>>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>>>>
>>>>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>>>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>> try {
>>>>>> // Try to reuse a previous connection
>>>>>> if (previousMxConnection == null) {
>>>>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>>>>> connection = connectionFactory.createConnection();
>>>>>> connection.setClientID(clientId);
>>>>>> connection.start();
>>>>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>>>>> destination = session.createTopic(topicName);
>>>>>> producer = session.createProducer(destination);
>>>>>> producer.setDeliveryMode(messageMode);
>>>>>>
>>>>>> mxConnection = new MatrixConnection();
>>>>>> mxConnection.connection = connection;
>>>>>> mxConnection.session = session;
>>>>>> mxConnection.destination = destination;
>>>>>> mxConnection.producer = producer;
>>>>>> }
>>>>>> else {
>>>>>> session = previousMxConnection.session;
>>>>>> if (previousMxConnection.producer == null) {
>>>>>> producer = session.createProducer(previousMxConnection.destination);
>>>>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>>> previousMxConnection.producer = producer;
>>>>>> }
>>>>>> else {
>>>>>> producer = previousMxConnection.producer;
>>>>>> }
>>>>>> mxConnection = previousMxConnection;
>>>>>> }
>>>>>>
>>>>>> TextMessage message = session.createTextMessage(messageText);
>>>>>> producer.send(message);
>>>>>>
>>>>>> return mxConnection;
>>>>>> }
>>>>>> catch(Exception e) {
>>>>>> throw new MatrixClientException(e);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>>>>
>>>>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>>>>
>>>>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>>>>
>>>>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>>>>
>>>>>>> Alistair
>>>>>>>
>>>>>>> --
>>>>>>> mov eax,1
>>>>>>> mov ebx,0
>>>>>>> int 80h
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>>>>
>>>>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>>>>
>>>>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>>>>> http://activemq.apache.org/async-sends.html
>>>>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>>>>
>>>>>>>> I'll try this on the broker connection to see if it helps:
>>>>>>>>
>>>>>>>> jms.useAsyncSend=false
>>>>>>>>
>>>>>>>> Alistair
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> mov eax,1
>>>>>>>> mov ebx,0
>>>>>>>> int 80h
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>>>>
>>>>>>>>> What is the exact broker version used on your side?
>>>>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>>>>
>>>>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>>>>
>>>>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>>>>
>>>>>>>>>> Alistair
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> mov eax,1
>>>>>>>>>> mov ebx,0
>>>>>>>>>> int 80h
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://fusesource.com
>>>>> http://blog.garytully.com
>>>>
>>>
>>>
>>
>>
>>
>> --
>> http://fusesource.com
>> http://blog.garytully.com
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
the topic consumers don't need to be present when the message is sent as the messages are persistent and they are durable topic subscribers. They are never persisted by activemq. They just disappear. I can't track the messages through the system as isAllowedToConsume is never called for the original message. It's only called when camel reroutes the message and camel never sees the missing messages, as verified by org.apache.camel.Processor instances.
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 27 Sep 2011, at 11:25, Gary Tully wrote:
> please provide some code that demonstrates your problem. Are you sure
> that the topic consumers are present when the messages are sent?
> Try and reduce the complexity of your usecase, the bottom line is that
> messages won't just disappear without good reason, there are hundreds
> of test cases that verify this is the case.
>
>
> On 27 September 2011 11:16, Alistair Young <al...@uhi.ac.uk> wrote:
>> not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
>>
>> message -> topicA -> camel -> topicB
>>
>> isAllowedToConsume is never called for topicA
>> isAllowedToConsume is called for topicB with a destination of topic://topicB
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 26 Sep 2011, at 16:19, Alistair Young wrote:
>>
>>> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>>>
>>> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>>>
>>> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>>>
>>> so on one server the offset and length are wrong as it seems to report too many chars
>>>
>>> Alistair
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>>>
>>>> Is this code shared by multiple threads? does it need synchronization?
>>>>
>>>> What you are experiencing does seem odd, it would be great if you
>>>> could provide a simple junit test case that can reproduce.
>>>>
>>>> Also, peeking that the code of the logging plugin, it should be
>>>> logging sends, that is again odd:
>>>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>>>
>>>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>>>
>>>>> thanks,
>>>>>
>>>>> Alistair
>>>>>
>>>>> try {
>>>>> // Try to reuse a previous connection
>>>>> if (previousMxConnection == null) {
>>>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>>>> connection = connectionFactory.createConnection();
>>>>> connection.setClientID(clientId);
>>>>> connection.start();
>>>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>>>> destination = session.createTopic(topicName);
>>>>> producer = session.createProducer(destination);
>>>>> producer.setDeliveryMode(messageMode);
>>>>>
>>>>> mxConnection = new MatrixConnection();
>>>>> mxConnection.connection = connection;
>>>>> mxConnection.session = session;
>>>>> mxConnection.destination = destination;
>>>>> mxConnection.producer = producer;
>>>>> }
>>>>> else {
>>>>> session = previousMxConnection.session;
>>>>> if (previousMxConnection.producer == null) {
>>>>> producer = session.createProducer(previousMxConnection.destination);
>>>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>>> previousMxConnection.producer = producer;
>>>>> }
>>>>> else {
>>>>> producer = previousMxConnection.producer;
>>>>> }
>>>>> mxConnection = previousMxConnection;
>>>>> }
>>>>>
>>>>> TextMessage message = session.createTextMessage(messageText);
>>>>> producer.send(message);
>>>>>
>>>>> return mxConnection;
>>>>> }
>>>>> catch(Exception e) {
>>>>> throw new MatrixClientException(e);
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>>>
>>>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>>>
>>>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>>>
>>>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>>>
>>>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>>>
>>>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>>>> http://activemq.apache.org/async-sends.html
>>>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>>>
>>>>>>> I'll try this on the broker connection to see if it helps:
>>>>>>>
>>>>>>> jms.useAsyncSend=false
>>>>>>>
>>>>>>> Alistair
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> mov eax,1
>>>>>>> mov ebx,0
>>>>>>> int 80h
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>>>
>>>>>>>> What is the exact broker version used on your side?
>>>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>>>
>>>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>>>
>>>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>>>
>>>>>>>>> Alistair
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> mov eax,1
>>>>>>>>> mov ebx,0
>>>>>>>>> int 80h
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> http://fusesource.com
>>>> http://blog.garytully.com
>>>
>>
>>
>
>
>
> --
> http://fusesource.com
> http://blog.garytully.com
Re: Random persistent messages lost
Posted by Gary Tully <ga...@gmail.com>.
please provide some code that demonstrates your problem. Are you sure
that the topic consumers are present when the messages are sent?
Try and reduce the complexity of your usecase, the bottom line is that
messages won't just disappear without good reason, there are hundreds
of test cases that verify this is the case.
On 27 September 2011 11:16, Alistair Young <al...@uhi.ac.uk> wrote:
> not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
>
> message -> topicA -> camel -> topicB
>
> isAllowedToConsume is never called for topicA
> isAllowedToConsume is called for topicB with a destination of topic://topicB
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 26 Sep 2011, at 16:19, Alistair Young wrote:
>
>> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>>
>> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>>
>> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>>
>> so on one server the offset and length are wrong as it seems to report too many chars
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>>
>>> Is this code shared by multiple threads? does it need synchronization?
>>>
>>> What you are experiencing does seem odd, it would be great if you
>>> could provide a simple junit test case that can reproduce.
>>>
>>> Also, peeking that the code of the logging plugin, it should be
>>> logging sends, that is again odd:
>>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>>
>>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>>
>>>> thanks,
>>>>
>>>> Alistair
>>>>
>>>> try {
>>>> // Try to reuse a previous connection
>>>> if (previousMxConnection == null) {
>>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>>> connection = connectionFactory.createConnection();
>>>> connection.setClientID(clientId);
>>>> connection.start();
>>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>>> destination = session.createTopic(topicName);
>>>> producer = session.createProducer(destination);
>>>> producer.setDeliveryMode(messageMode);
>>>>
>>>> mxConnection = new MatrixConnection();
>>>> mxConnection.connection = connection;
>>>> mxConnection.session = session;
>>>> mxConnection.destination = destination;
>>>> mxConnection.producer = producer;
>>>> }
>>>> else {
>>>> session = previousMxConnection.session;
>>>> if (previousMxConnection.producer == null) {
>>>> producer = session.createProducer(previousMxConnection.destination);
>>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>>> previousMxConnection.producer = producer;
>>>> }
>>>> else {
>>>> producer = previousMxConnection.producer;
>>>> }
>>>> mxConnection = previousMxConnection;
>>>> }
>>>>
>>>> TextMessage message = session.createTextMessage(messageText);
>>>> producer.send(message);
>>>>
>>>> return mxConnection;
>>>> }
>>>> catch(Exception e) {
>>>> throw new MatrixClientException(e);
>>>> }
>>>> }
>>>>
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>>
>>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>>
>>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>>
>>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>>
>>>>> Alistair
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>>
>>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>>
>>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>>> http://activemq.apache.org/async-sends.html
>>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>>
>>>>>> I'll try this on the broker connection to see if it helps:
>>>>>>
>>>>>> jms.useAsyncSend=false
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>>
>>>>>>> What is the exact broker version used on your side?
>>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>>
>>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>>
>>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>>
>>>>>>>> Alistair
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> mov eax,1
>>>>>>>> mov ebx,0
>>>>>>>> int 80h
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> http://fusesource.com
>>> http://blog.garytully.com
>>
>
>
--
http://fusesource.com
http://blog.garytully.com
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
not a lot seems to work. 8 identical messages sent to the broker in quick succession. Three arrived, 5 just disappeared. It's complicated by the fact MessageAuthorizationPolicy::isAllowedToConsume never seems to be called for the original message, so I can't really verify whether the message disappeared once inside activemq. isAllowedToConsume is only called when the message is routed by camel to the next topic:
message -> topicA -> camel -> topicB
isAllowedToConsume is never called for topicA
isAllowedToConsume is called for topicB with a destination of topic://topicB
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 26 Sep 2011, at 16:19, Alistair Young wrote:
> trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
>
> public boolean isAllowedToConsume(ConnectionContext context, Message message) {
>
> new String(message.getContent().data, message.getContent().offset, message.getContent().length)
>
> so on one server the offset and length are wrong as it seems to report too many chars
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 22 Sep 2011, at 12:05, Gary Tully wrote:
>
>> Is this code shared by multiple threads? does it need synchronization?
>>
>> What you are experiencing does seem odd, it would be great if you
>> could provide a simple junit test case that can reproduce.
>>
>> Also, peeking that the code of the logging plugin, it should be
>> logging sends, that is again odd:
>> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>>
>> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>>
>>> thanks,
>>>
>>> Alistair
>>>
>>> try {
>>> // Try to reuse a previous connection
>>> if (previousMxConnection == null) {
>>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>>> connection = connectionFactory.createConnection();
>>> connection.setClientID(clientId);
>>> connection.start();
>>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>>> destination = session.createTopic(topicName);
>>> producer = session.createProducer(destination);
>>> producer.setDeliveryMode(messageMode);
>>>
>>> mxConnection = new MatrixConnection();
>>> mxConnection.connection = connection;
>>> mxConnection.session = session;
>>> mxConnection.destination = destination;
>>> mxConnection.producer = producer;
>>> }
>>> else {
>>> session = previousMxConnection.session;
>>> if (previousMxConnection.producer == null) {
>>> producer = session.createProducer(previousMxConnection.destination);
>>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>> previousMxConnection.producer = producer;
>>> }
>>> else {
>>> producer = previousMxConnection.producer;
>>> }
>>> mxConnection = previousMxConnection;
>>> }
>>>
>>> TextMessage message = session.createTextMessage(messageText);
>>> producer.send(message);
>>>
>>> return mxConnection;
>>> }
>>> catch(Exception e) {
>>> throw new MatrixClientException(e);
>>> }
>>> }
>>>
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>>
>>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>>
>>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>>
>>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>>
>>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>>
>>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>>> http://activemq.apache.org/async-sends.html
>>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>>
>>>>> I'll try this on the broker connection to see if it helps:
>>>>>
>>>>> jms.useAsyncSend=false
>>>>>
>>>>> Alistair
>>>>>
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>>
>>>>>> What is the exact broker version used on your side?
>>>>>> In case its not the latest released version, can you try the latest version?
>>>>>>
>>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>>
>>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>>
>>>>>>> Alistair
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> mov eax,1
>>>>>>> mov ebx,0
>>>>>>> int 80h
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>> --
>> http://fusesource.com
>> http://blog.garytully.com
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
trying to log incoming messages at the broker to narrow the problem and another problem appears. This works on one server but adds 0x0 chars on another so the message fails to parse:
public boolean isAllowedToConsume(ConnectionContext context, Message message) {
new String(message.getContent().data, message.getContent().offset, message.getContent().length)
so on one server the offset and length are wrong as it seems to report too many chars
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 22 Sep 2011, at 12:05, Gary Tully wrote:
> Is this code shared by multiple threads? does it need synchronization?
>
> What you are experiencing does seem odd, it would be great if you
> could provide a simple junit test case that can reproduce.
>
> Also, peeking that the code of the logging plugin, it should be
> logging sends, that is again odd:
> http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
>
> On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
>> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>>
>> thanks,
>>
>> Alistair
>>
>> try {
>> // Try to reuse a previous connection
>> if (previousMxConnection == null) {
>> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
>> connection = connectionFactory.createConnection();
>> connection.setClientID(clientId);
>> connection.start();
>> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>> destination = session.createTopic(topicName);
>> producer = session.createProducer(destination);
>> producer.setDeliveryMode(messageMode);
>>
>> mxConnection = new MatrixConnection();
>> mxConnection.connection = connection;
>> mxConnection.session = session;
>> mxConnection.destination = destination;
>> mxConnection.producer = producer;
>> }
>> else {
>> session = previousMxConnection.session;
>> if (previousMxConnection.producer == null) {
>> producer = session.createProducer(previousMxConnection.destination);
>> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>> previousMxConnection.producer = producer;
>> }
>> else {
>> producer = previousMxConnection.producer;
>> }
>> mxConnection = previousMxConnection;
>> }
>>
>> TextMessage message = session.createTextMessage(messageText);
>> producer.send(message);
>>
>> return mxConnection;
>> }
>> catch(Exception e) {
>> throw new MatrixClientException(e);
>> }
>> }
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>
>>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>>
>>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>>
>>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>>
>>> Alistair
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>
>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>>
>>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>>> http://activemq.apache.org/async-sends.html
>>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>>
>>>> I'll try this on the broker connection to see if it helps:
>>>>
>>>> jms.useAsyncSend=false
>>>>
>>>> Alistair
>>>>
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>
>>>>> What is the exact broker version used on your side?
>>>>> In case its not the latest released version, can you try the latest version?
>>>>>
>>>>> Do you set any particular headers on the Topic message in your producer?
>>>>>
>>>>>
>>>>>
>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>
>>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>
>>>
>>
>>
>
>
>
> --
> http://fusesource.com
> http://blog.garytully.com
Re: Random persistent messages lost
Posted by Gary Tully <ga...@gmail.com>.
Is this code shared by multiple threads? does it need synchronization?
What you are experiencing does seem odd, it would be great if you
could provide a simple junit test case that can reproduce.
Also, peeking that the code of the logging plugin, it should be
logging sends, that is again odd:
http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/src/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=markup
On 22 September 2011 10:59, Alistair Young <al...@uhi.ac.uk> wrote:
> would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
>
> thanks,
>
> Alistair
>
> try {
> // Try to reuse a previous connection
> if (previousMxConnection == null) {
> ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
> connection = connectionFactory.createConnection();
> connection.setClientID(clientId);
> connection.start();
> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
> destination = session.createTopic(topicName);
> producer = session.createProducer(destination);
> producer.setDeliveryMode(messageMode);
>
> mxConnection = new MatrixConnection();
> mxConnection.connection = connection;
> mxConnection.session = session;
> mxConnection.destination = destination;
> mxConnection.producer = producer;
> }
> else {
> session = previousMxConnection.session;
> if (previousMxConnection.producer == null) {
> producer = session.createProducer(previousMxConnection.destination);
> producer.setDeliveryMode(DeliveryMode.PERSISTENT);
> previousMxConnection.producer = producer;
> }
> else {
> producer = previousMxConnection.producer;
> }
> mxConnection = previousMxConnection;
> }
>
> TextMessage message = session.createTextMessage(messageText);
> producer.send(message);
>
> return mxConnection;
> }
> catch(Exception e) {
> throw new MatrixClientException(e);
> }
> }
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>
>> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>>
>> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>>
>> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>
>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>>
>>> there's quite a high rate of packet loss on the network it seems and according to this:
>>> http://activemq.apache.org/async-sends.html
>>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>>
>>> I'll try this on the broker connection to see if it helps:
>>>
>>> jms.useAsyncSend=false
>>>
>>> Alistair
>>>
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>
>>>> What is the exact broker version used on your side?
>>>> In case its not the latest released version, can you try the latest version?
>>>>
>>>> Do you set any particular headers on the Topic message in your producer?
>>>>
>>>>
>>>>
>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>
>>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>>
>>>>> Alistair
>>>>>
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>
>>
>
>
--
http://fusesource.com
http://blog.garytully.com
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
would someone be kind enough to have a quick look at this code to see if it could cause problems? The method returns a reusable connection so the user of the class doesn't have to connect/disconnect on every message. The first message never fails, it's always when reusing the connection that random messages disappear.
thanks,
Alistair
try {
// Try to reuse a previous connection
if (previousMxConnection == null) {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerURL);
connection = connectionFactory.createConnection();
connection.setClientID(clientId);
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
destination = session.createTopic(topicName);
producer = session.createProducer(destination);
producer.setDeliveryMode(messageMode);
mxConnection = new MatrixConnection();
mxConnection.connection = connection;
mxConnection.session = session;
mxConnection.destination = destination;
mxConnection.producer = producer;
}
else {
session = previousMxConnection.session;
if (previousMxConnection.producer == null) {
producer = session.createProducer(previousMxConnection.destination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
previousMxConnection.producer = producer;
}
else {
producer = previousMxConnection.producer;
}
mxConnection = previousMxConnection;
}
TextMessage message = session.createTextMessage(messageText);
producer.send(message);
return mxConnection;
}
catch(Exception e) {
throw new MatrixClientException(e);
}
}
--
mov eax,1
mov ebx,0
int 80h
On 22 Sep 2011, at 10:50, Alistair Young wrote:
> nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
>
> <loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
>
> I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>
>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>>
>> there's quite a high rate of packet loss on the network it seems and according to this:
>> http://activemq.apache.org/async-sends.html
>> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>>
>> I'll try this on the broker connection to see if it helps:
>>
>> jms.useAsyncSend=false
>>
>> Alistair
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>
>>> What is the exact broker version used on your side?
>>> In case its not the latest released version, can you try the latest version?
>>>
>>> Do you set any particular headers on the Topic message in your producer?
>>>
>>>
>>>
>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>
>>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>>
>>>> Alistair
>>>>
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
nothing seems to help. There are still messages just disappearing. Also the logging doesn't seem to log anything to do with producers:
<loggingBrokerPlugin logAll="true" logConnectionEvents="true" />
I can see lots of camel consumers from the local machine looking at the routes but nothing about incoming messages from producers.
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 21 Sep 2011, at 13:13, Alistair Young wrote:
> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
>
> there's quite a high rate of packet loss on the network it seems and according to this:
> http://activemq.apache.org/async-sends.html
> the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
>
> I'll try this on the broker connection to see if it helps:
>
> jms.useAsyncSend=false
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>
>> What is the exact broker version used on your side?
>> In case its not the latest released version, can you try the latest version?
>>
>> Do you set any particular headers on the Topic message in your producer?
>>
>>
>>
>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>
>>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>>
>>> Alistair
>>>
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
I'm running ActiveMQ 5.5.0 as a spring webapp rather than the standalone version. Would you recommend the standalone one? Not sure how to use camel with that though.
there's quite a high rate of packet loss on the network it seems and according to this:
http://activemq.apache.org/async-sends.html
the default is sync send for persistent messages outside transactions. I'm using transactions on the broker but not the producer to preserve camel routes across restarts. So I presume the producer is using sync mode to send the messages as they are persistent. However, no error from the broker and no message arriving might hint at async mode falling foul of packet loss perhaps?
I'll try this on the broker connection to see if it helps:
jms.useAsyncSend=false
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
> What is the exact broker version used on your side?
> In case its not the latest released version, can you try the latest version?
>
> Do you set any particular headers on the Topic message in your producer?
>
>
>
> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>
>> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>>
>> Alistair
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
Re: Random persistent messages lost
Posted by Torsten Mielke <to...@fusesource.com>.
What is the exact broker version used on your side?
In case its not the latest released version, can you try the latest version?
Do you set any particular headers on the Topic message in your producer?
On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
> I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
I can't explain this at all. It's almost like the opposite of reliable messaging. At times, almost 1 in 3 messages just disappears. No errors. The KahaDB/db-*.log show no record of the message every arriving and yet the producer doesn't get an error.
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 21 Sep 2011, at 12:26, Alistair Young wrote:
> yep, there is a durable topic consumer waiting for them
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 21 Sep 2011, at 12:20, Gary Tully wrote:
>
>> topic messages are only persisted to the store if there are durable
>> consumers/subscriptions
>>
>> On 21 September 2011 11:04, Alistair Young <al...@uhi.ac.uk> wrote:
>>> I should say, the messages go to a topic on the broker and are then routed by camel to another topc. Would camel be dropping the messages? Would they not be logged in the kahadb store no matter what though?
>>>
>>> Alistair
>>>
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 21 Sep 2011, at 10:25, Alistair Young wrote:
>>>
>>>> I doubt it. In a batch of three messages sent in 2 seconds, one disappeard. Is there a default throttle limit perhaps on the broker? But surely that would cause the producer to error out or something?
>>>>
>>>> ------------------------------------
>>>> Alistair Young
>>>> Àrd-Innleadair air Bathar-Bog
>>>> UHI@Sabhal Mòr Ostaig
>>>>
>>>>
>>>>
>>>>
>>>> On 21 Sep 2011, at 09:59, Torsten Mielke wrote:
>>>>
>>>>> Any chance you can reproduce this behavior in a little JUnit test case?
>>>>>
>>>>>
>>>>>
>>>>> On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>>>>>
>>>>>>> There was no producer, broker or connection crash while the msg was in transit?
>>>>>> no, the logs show it was up and sending
>>>>>>
>>>>>>> The msg does not have an expiration time?
>>>>>>
>>>>>> no
>>>>>>
>>>>>>> Are you certain that no consumer has got the msg already?
>>>>>>
>>>>>> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>>>>>>
>>>>>> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>>>>>>
>>>>>> ------------------------------------
>>>>>> Alistair Young
>>>>>> Àrd-Innleadair air Bathar-Bog
>>>>>> UHI@Sabhal Mòr Ostaig
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>> --
>> http://fusesource.com
>> http://blog.garytully.com
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
yep, there is a durable topic consumer waiting for them
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 21 Sep 2011, at 12:20, Gary Tully wrote:
> topic messages are only persisted to the store if there are durable
> consumers/subscriptions
>
> On 21 September 2011 11:04, Alistair Young <al...@uhi.ac.uk> wrote:
>> I should say, the messages go to a topic on the broker and are then routed by camel to another topc. Would camel be dropping the messages? Would they not be logged in the kahadb store no matter what though?
>>
>> Alistair
>>
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 21 Sep 2011, at 10:25, Alistair Young wrote:
>>
>>> I doubt it. In a batch of three messages sent in 2 seconds, one disappeard. Is there a default throttle limit perhaps on the broker? But surely that would cause the producer to error out or something?
>>>
>>> ------------------------------------
>>> Alistair Young
>>> Àrd-Innleadair air Bathar-Bog
>>> UHI@Sabhal Mòr Ostaig
>>>
>>>
>>>
>>>
>>> On 21 Sep 2011, at 09:59, Torsten Mielke wrote:
>>>
>>>> Any chance you can reproduce this behavior in a little JUnit test case?
>>>>
>>>>
>>>>
>>>> On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>>>>
>>>>>> There was no producer, broker or connection crash while the msg was in transit?
>>>>> no, the logs show it was up and sending
>>>>>
>>>>>> The msg does not have an expiration time?
>>>>>
>>>>> no
>>>>>
>>>>>> Are you certain that no consumer has got the msg already?
>>>>>
>>>>> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>>>>>
>>>>> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>>>>>
>>>>> ------------------------------------
>>>>> Alistair Young
>>>>> Àrd-Innleadair air Bathar-Bog
>>>>> UHI@Sabhal Mòr Ostaig
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>
>
>
> --
> http://fusesource.com
> http://blog.garytully.com
Re: Random persistent messages lost
Posted by Gary Tully <ga...@gmail.com>.
topic messages are only persisted to the store if there are durable
consumers/subscriptions
On 21 September 2011 11:04, Alistair Young <al...@uhi.ac.uk> wrote:
> I should say, the messages go to a topic on the broker and are then routed by camel to another topc. Would camel be dropping the messages? Would they not be logged in the kahadb store no matter what though?
>
> Alistair
>
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 21 Sep 2011, at 10:25, Alistair Young wrote:
>
>> I doubt it. In a batch of three messages sent in 2 seconds, one disappeard. Is there a default throttle limit perhaps on the broker? But surely that would cause the producer to error out or something?
>>
>> ------------------------------------
>> Alistair Young
>> Àrd-Innleadair air Bathar-Bog
>> UHI@Sabhal Mòr Ostaig
>>
>>
>>
>>
>> On 21 Sep 2011, at 09:59, Torsten Mielke wrote:
>>
>>> Any chance you can reproduce this behavior in a little JUnit test case?
>>>
>>>
>>>
>>> On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>>>
>>>>> There was no producer, broker or connection crash while the msg was in transit?
>>>> no, the logs show it was up and sending
>>>>
>>>>> The msg does not have an expiration time?
>>>>
>>>> no
>>>>
>>>>> Are you certain that no consumer has got the msg already?
>>>>
>>>> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>>>>
>>>> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>>>>
>>>> ------------------------------------
>>>> Alistair Young
>>>> Àrd-Innleadair air Bathar-Bog
>>>> UHI@Sabhal Mòr Ostaig
>>>
>>>
>>>
>>>
>>
>
>
--
http://fusesource.com
http://blog.garytully.com
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
I should say, the messages go to a topic on the broker and are then routed by camel to another topc. Would camel be dropping the messages? Would they not be logged in the kahadb store no matter what though?
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 21 Sep 2011, at 10:25, Alistair Young wrote:
> I doubt it. In a batch of three messages sent in 2 seconds, one disappeard. Is there a default throttle limit perhaps on the broker? But surely that would cause the producer to error out or something?
>
> ------------------------------------
> Alistair Young
> Àrd-Innleadair air Bathar-Bog
> UHI@Sabhal Mòr Ostaig
>
>
>
>
> On 21 Sep 2011, at 09:59, Torsten Mielke wrote:
>
>> Any chance you can reproduce this behavior in a little JUnit test case?
>>
>>
>>
>> On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>>
>>>> There was no producer, broker or connection crash while the msg was in transit?
>>> no, the logs show it was up and sending
>>>
>>>> The msg does not have an expiration time?
>>>
>>> no
>>>
>>>> Are you certain that no consumer has got the msg already?
>>>
>>> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>>>
>>> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>>>
>>> ------------------------------------
>>> Alistair Young
>>> Àrd-Innleadair air Bathar-Bog
>>> UHI@Sabhal Mòr Ostaig
>>
>>
>>
>>
>
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
I doubt it. In a batch of three messages sent in 2 seconds, one disappeard. Is there a default throttle limit perhaps on the broker? But surely that would cause the producer to error out or something?
------------------------------------
Alistair Young
Àrd-Innleadair air Bathar-Bog
UHI@Sabhal Mòr Ostaig
On 21 Sep 2011, at 09:59, Torsten Mielke wrote:
> Any chance you can reproduce this behavior in a little JUnit test case?
>
>
>
> On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>
>>> There was no producer, broker or connection crash while the msg was in transit?
>> no, the logs show it was up and sending
>>
>>> The msg does not have an expiration time?
>>
>> no
>>
>>> Are you certain that no consumer has got the msg already?
>>
>> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>>
>> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>>
>> ------------------------------------
>> Alistair Young
>> Àrd-Innleadair air Bathar-Bog
>> UHI@Sabhal Mòr Ostaig
>
>
>
>
Re: Random persistent messages lost
Posted by Torsten Mielke <to...@fusesource.com>.
Any chance you can reproduce this behavior in a little JUnit test case?
On Sep 21, 2011, at 10:15 AM, Alistair Young wrote:
>> There was no producer, broker or connection crash while the msg was in transit?
> no, the logs show it was up and sending
>
>> The msg does not have an expiration time?
>
> no
>
>> Are you certain that no consumer has got the msg already?
>
> yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
>
> the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
>
> ------------------------------------
> Alistair Young
> Àrd-Innleadair air Bathar-Bog
> UHI@Sabhal Mòr Ostaig
Re: Random persistent messages lost
Posted by Alistair Young <al...@uhi.ac.uk>.
> There was no producer, broker or connection crash while the msg was in transit?
no, the logs show it was up and sending
> The msg does not have an expiration time?
no
> Are you certain that no consumer has got the msg already?
yes, there's no record of it arriving at the consumer and no log of it in the kahadb data store
the producer sent 49 messages in 2 seconds.The 43rd and 46th messages disappeared. They never reached the broker it would seem.
------------------------------------
Alistair Young
Àrd-Innleadair air Bathar-Bog
UHI@Sabhal Mòr Ostaig
On 20 Sep 2011, at 15:50, Torsten Mielke wrote:
> There was no producer, broker or connection crash while the msg was in transit?
> The msg does not have an expiration time?
> Are you certain that no consumer has got the msg already?
>
> You could turn on full logging inside the broker, it will log each msgs being received. Might not be suitable for a production system that deals with a higher load of msgs.
>
>
> Torsten Mielke
> torsten@fusesource.com
> tmielke@blogspot.com
>
>
> On Sep 20, 2011, at 12:18 PM, Alistair Young wrote:
>
>> Is there anything that might cause a message to go missing? A producer sends a persistent message to a topic but now and then, although no error is returned from the broker, the message just disappears. It's not logged in the data store and it's as if it never reached the broker but that would have produced an error for the producer to catch.
>>
>> thanks,
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>
>
>
Re: Random persistent messages lost
Posted by Torsten Mielke <to...@fusesource.com>.
There was no producer, broker or connection crash while the msg was in transit?
The msg does not have an expiration time?
Are you certain that no consumer has got the msg already?
You could turn on full logging inside the broker, it will log each msgs being received. Might not be suitable for a production system that deals with a higher load of msgs.
Torsten Mielke
torsten@fusesource.com
tmielke@blogspot.com
On Sep 20, 2011, at 12:18 PM, Alistair Young wrote:
> Is there anything that might cause a message to go missing? A producer sends a persistent message to a topic but now and then, although no error is returned from the broker, the message just disappears. It's not logged in the data store and it's as if it never reached the broker but that would have produced an error for the producer to catch.
>
> thanks,
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>