You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by j0llyr0g3r <ti...@wincor-nixdorf.com> on 2008/04/10 11:42:32 UTC
Failed to send ..... to dead letter queue
Hey folks,
i really hope someone can help me out, since i am running out of ideas....
My problem seems quite simple:
-> I have a self-written message producer using AMQ-libraries
-> I have a self-written message consumer using AMQ-libraries
-> And a (single) AMQ-Broker in the middle
-> All used AMQ-libraries have the some version (version 5, downloaded one
week ago)
-> consumer and producer use "real" tcp-connections, _not_
"in-jvm"-connections, even when they are running on the same machine
Now, if all of the three components (producer + broker + consumer) run on
the same host, everything works _fine_....
The problem:
If all 3 components run on different hosts, nothing happens, meaning:
-> My producer "hangs":
INFO (SendSyncCmd.send) 2008-04-08 11:18:11,085
Sending message.....
INFO (SendSyncCmd.send) 2008-04-08 11:18:11,090
Waiting for reponse message...
My producer sends to the queue:
Publishing a Message with size 255 to queue: CMD.ESF-PROTO-2
-> My consumer prints nothing out, i.e. receives nothing.
But he listens on right queue: CMD.ESF-PROTO-2
-> Now to the interesting part: The broker.
If i take a look at the web-interface i see something like:
Name Number Of Pending Messages Number Of Consumers
CMD.ESF-PROTO-2 -1 1
Number Of Pending Messages = -1? What is this supposed to mean?
If i look at the command-line output of the broker i see:
WARN RegionBroker - Failed to send ActiveMQBytesMessage
{commandId = 7, responseRequired = false, messageId =
ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1:1,
originalDestination = null, originalTransactionId = null, producerId =
ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1, destination =
queue://CMD.ESF-PROTO-2, transactionId = null, expiration = 1207822630933,
timestamp = 1207822630678, arrival = 0, brokerInTime = 1209266957622,
brokerOutTime = 0, correlationId = null, replyTo =
temp-queue://ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1,
persistent = false, type = null, priority = 4, groupID = null, groupSequence
= 0, targetConsumerId = null, compressed = false, userID = null, content =
org.apache.activemq.util.ByteSequence@ee3dee, marshalledProperties = null,
dataStructure = null, redeliveryCounter = 0, size = 0, properties = null,
readOnlyProperties = false, readOnlyBody = false, droppable = false}
ActiveMQBytesMessage{ bytesOut = null, dataOut = null, dataIn = null } to
dead letter queue
So, in a nutshell, the broker tells me:
Failed to send ActiveMQBytesMessage ......to dead letter queue
But tells in the same output:
destination = queue://CMD.ESF-PROTO-2
which is correct! This queue exists and i got a consumer listening!
Now, if i look in the source-code of activemq, there is only one location
where this message is generated:
-> package org.apache.activemq.broker.region;
-> Class RegionBroker
-> Method sendToDeadLetterQueue()
public void sendToDeadLetterQueue(ConnectionContext context,
MessageReference node){
try{
boolean sent=false;
if(node!=null){
Message message=node.getMessage();
if(message!=null&&node.getRegionDestination()!=null){
DeadLetterStrategy deadLetterStrategy=node
.getRegionDestination().getDeadLetterStrategy();
if(deadLetterStrategy!=null){
if(deadLetterStrategy.isSendToDeadLetterQueue(message)){
long expiration=message.getExpiration();
message.setExpiration(0);
message.setProperty("originalExpiration",new Long(
expiration));
if(!message.isPersistent()){
message.setPersistent(true);
message.setProperty("originalDeliveryMode",
"NON_PERSISTENT");
}
// The original destination and transaction id do
// not get filled when the message is first
// sent,
// it is only populated if the message is routed to
// another destination like the DLQ
ActiveMQDestination deadLetterDestination=deadLetterStrategy
.getDeadLetterQueueFor(message
.getDestination());
if (context.getBroker()==null) {
context.setBroker(getRoot());
}
BrokerSupport.resend(context,message,
deadLetterDestination);
sent=true;
}
}
}
}
if(sent==false){
LOG.warn("Failed to send "+node+" to dead letter queue");
}
}catch(Exception e){
LOG.warn("Failed to pass expired message to dead letter queue",e);
}
}
To be more specific, the last "if":
if(sent==false){
LOG.warn("Failed to send "+node+" to dead letter queue");
}
Unfortunately, there is neither a java-doc nor a possibility to increase the
logging-output, so i am clueless why "sent=false" is true.
If you look at the source-code above, there are four possibilites:
node=null || message=null || node.getRegionDestination()=null ||
deadLetterStrategy=null
And that's where i am stuck....
I would really appreciate it, if some of you guys could help me out......
--
View this message in context: http://www.nabble.com/Failed-to-send-.....-to-dead-letter-queue-tp16606547s2354p16606547.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Failed to send ..... to dead letter queue
Posted by j0llyr0g3r <ti...@wincor-nixdorf.com>.
Hey ho,
i finally solved the problem and i consider this an activemq bug:
The reason for the error:
-> Broker was running on Java 1.5
-> Producer was running on Java 1.6
Solution:
Don't use Java 1.5 and Java 1.6 together.
This is a freakin' catastrophy, because nothing, really nothing in the
"warn" message gives you an idea that this could be related in any way to
different java version.
And i really think that the same AMQ-versions should work together
regardless wether Java 1.5 or Java 1.6. is used, so i consider this a bug.
Any developers here? What your opinion on that?
j0llyr0g3r wrote:
>
> Hey folks,
>
> i really hope someone can help me out, since i am running out of ideas....
>
> My problem seems quite simple:
>
> -> I have a self-written message producer using AMQ-libraries
> -> I have a self-written message consumer using AMQ-libraries
> -> And a (single) AMQ-Broker in the middle
> -> All used AMQ-libraries have the some version (version 5, downloaded one
> week ago)
> -> consumer and producer use "real" tcp-connections, _not_
> "in-jvm"-connections, even when they are running on the same machine
>
> Now, if all of the three components (producer + broker + consumer) run on
> the same host, everything works _fine_....
>
> The problem:
>
> If all 3 components run on different hosts, nothing happens, meaning:
>
> -> My producer "hangs":
>
> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,085
> Sending message.....
> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,090
> Waiting for reponse message...
>
> My producer sends to the queue:
>
> Publishing a Message with size 255 to queue: CMD.ESF-PROTO-2
>
> -> My consumer prints nothing out, i.e. receives nothing.
>
> But he listens on right queue: CMD.ESF-PROTO-2
>
> -> Now to the interesting part: The broker.
>
> If i take a look at the web-interface i see something like:
>
> Name Number Of Pending Messages Number Of Consumers
>
> CMD.ESF-PROTO-2 -1 1
>
> Number Of Pending Messages = -1? What is this supposed to mean?
>
> If i look at the command-line output of the broker i see:
>
> WARN RegionBroker - Failed to send ActiveMQBytesMessage
> {commandId = 7, responseRequired = false, messageId =
> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1:1,
> originalDestination = null, originalTransactionId = null, producerId =
> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1, destination =
> queue://CMD.ESF-PROTO-2, transactionId = null, expiration = 1207822630933,
> timestamp = 1207822630678, arrival = 0, brokerInTime = 1209266957622,
> brokerOutTime = 0, correlationId = null, replyTo =
> temp-queue://ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1,
> persistent = false, type = null, priority = 4, groupID = null,
> groupSequence = 0, targetConsumerId = null, compressed = false, userID =
> null, content = org.apache.activemq.util.ByteSequence@ee3dee,
> marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
> size = 0, properties = null, readOnlyProperties = false, readOnlyBody =
> false, droppable = false} ActiveMQBytesMessage{ bytesOut = null, dataOut =
> null, dataIn = null } to dead letter queue
>
> So, in a nutshell, the broker tells me:
>
> Failed to send ActiveMQBytesMessage ......to dead letter queue
>
> But tells in the same output:
>
> destination = queue://CMD.ESF-PROTO-2
>
> which is correct! This queue exists and i got a consumer listening!
>
> Now, if i look in the source-code of activemq, there is only one location
> where this message is generated:
>
> -> package org.apache.activemq.broker.region;
> -> Class RegionBroker
> -> Method sendToDeadLetterQueue()
>
> public void sendToDeadLetterQueue(ConnectionContext context,
> MessageReference node){
> try{
> boolean sent=false;
> if(node!=null){
> Message message=node.getMessage();
> if(message!=null&&node.getRegionDestination()!=null){
> DeadLetterStrategy deadLetterStrategy=node
> .getRegionDestination().getDeadLetterStrategy();
> if(deadLetterStrategy!=null){
> if(deadLetterStrategy.isSendToDeadLetterQueue(message)){
> long expiration=message.getExpiration();
> message.setExpiration(0);
> message.setProperty("originalExpiration",new Long(
> expiration));
> if(!message.isPersistent()){
> message.setPersistent(true);
> message.setProperty("originalDeliveryMode",
> "NON_PERSISTENT");
> }
> // The original destination and transaction id do
> // not get filled when the message is first
> // sent,
> // it is only populated if the message is routed to
> // another destination like the DLQ
> ActiveMQDestination deadLetterDestination=deadLetterStrategy
> .getDeadLetterQueueFor(message
> .getDestination());
> if (context.getBroker()==null) {
> context.setBroker(getRoot());
> }
> BrokerSupport.resend(context,message,
> deadLetterDestination);
> sent=true;
> }
> }
> }
> }
> if(sent==false){
> LOG.warn("Failed to send "+node+" to dead letter queue");
> }
> }catch(Exception e){
> LOG.warn("Failed to pass expired message to dead letter queue",e);
> }
> }
>
> To be more specific, the last "if":
>
> if(sent==false){
> LOG.warn("Failed to send "+node+" to dead letter queue");
> }
>
>
> Unfortunately, there is neither a java-doc nor a possibility to increase
> the logging-output, so i am clueless why "sent=false" is true.
>
> If you look at the source-code above, there are four possibilites:
>
> node=null || message=null || node.getRegionDestination()=null ||
> deadLetterStrategy=null
>
> And that's where i am stuck....
>
> I would really appreciate it, if some of you guys could help me out......
>
--
View this message in context: http://www.nabble.com/Failed-to-send-.....-to-dead-letter-queue-tp16606547s2354p16608580.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Failed to send ..... to dead letter queue
Posted by Rob Davies <ra...@gmail.com>.
Was just looking into this - so thanks for the update!
On 10 Apr 2008, at 16:00, j0llyr0g3r wrote:
>
> Update again:
>
> i have to correct myself: This is NOT an AMQ / Java problem, but a
> problem
> with SLES9. (All of my testservers run on sles9, that's why i
> thought it
> would be an AMQ / Java problem)
>
> I can reproduce above described behaviour:
>
> Producer on Ubuntu / Java 1.6 -> Broker on Sles 9 / Java 1.6 ->
> Consumer
> on Sles 9 / Java 1.6
>
> => NOT working!
>
> If i substitute "SLES 9" with any other distribution (tested so far:
> Ubuntu
> and gentoo) + java 1.6
>
> => working!
>
> I really don't know what the hell SLES 9 is doing, and i don't care,
> because
> the first thing i will do is to remove this shitty product
> immediately from
> all of my testservers. (Not the first time SLES 9 is a pain in the
> ass in
> comparison to other distributions)
>
>
> j0llyr0g3r wrote:
>>
>> Hey folks,
>>
>> i really hope someone can help me out, since i am running out of
>> ideas....
>>
>> My problem seems quite simple:
>>
>> -> I have a self-written message producer using AMQ-libraries
>> -> I have a self-written message consumer using AMQ-libraries
>> -> And a (single) AMQ-Broker in the middle
>> -> All used AMQ-libraries have the some version (version 5,
>> downloaded one
>> week ago)
>> -> consumer and producer use "real" tcp-connections, _not_
>> "in-jvm"-connections, even when they are running on the same machine
>>
>> Now, if all of the three components (producer + broker + consumer)
>> run on
>> the same host, everything works _fine_....
>>
>> The problem:
>>
>> If all 3 components run on different hosts, nothing happens, meaning:
>>
>> -> My producer "hangs":
>>
>> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,085
>> Sending message.....
>> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,090
>> Waiting for reponse message...
>>
>> My producer sends to the queue:
>>
>> Publishing a Message with size 255 to queue: CMD.ESF-PROTO-2
>>
>> -> My consumer prints nothing out, i.e. receives nothing.
>>
>> But he listens on right queue: CMD.ESF-PROTO-2
>>
>> -> Now to the interesting part: The broker.
>>
>> If i take a look at the web-interface i see something like:
>>
>> Name Number Of Pending Messages Number Of Consumers
>>
>> CMD.ESF-PROTO-2 -1 1
>>
>> Number Of Pending Messages = -1? What is this supposed to mean?
>>
>> If i look at the command-line output of the broker i see:
>>
>> WARN RegionBroker - Failed to send
>> ActiveMQBytesMessage
>> {commandId = 7, responseRequired = false, messageId =
>> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1:1,
>> originalDestination = null, originalTransactionId = null,
>> producerId =
>> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1,
>> destination =
>> queue://CMD.ESF-PROTO-2, transactionId = null, expiration =
>> 1207822630933,
>> timestamp = 1207822630678, arrival = 0, brokerInTime = 1209266957622,
>> brokerOutTime = 0, correlationId = null, replyTo =
>> temp-queue://ID:sdoesmon.wincor-
>> nixdorf.com-41873-1207822630035-0:0:1,
>> persistent = false, type = null, priority = 4, groupID = null,
>> groupSequence = 0, targetConsumerId = null, compressed = false,
>> userID =
>> null, content = org.apache.activemq.util.ByteSequence@ee3dee,
>> marshalledProperties = null, dataStructure = null,
>> redeliveryCounter = 0,
>> size = 0, properties = null, readOnlyProperties = false,
>> readOnlyBody =
>> false, droppable = false} ActiveMQBytesMessage{ bytesOut = null,
>> dataOut =
>> null, dataIn = null } to dead letter queue
>>
>> So, in a nutshell, the broker tells me:
>>
>> Failed to send ActiveMQBytesMessage ......to dead letter queue
>>
>> But tells in the same output:
>>
>> destination = queue://CMD.ESF-PROTO-2
>>
>> which is correct! This queue exists and i got a consumer listening!
>>
>> Now, if i look in the source-code of activemq, there is only one
>> location
>> where this message is generated:
>>
>> -> package org.apache.activemq.broker.region;
>> -> Class RegionBroker
>> -> Method sendToDeadLetterQueue()
>>
>> public void sendToDeadLetterQueue(ConnectionContext context,
>> MessageReference node){
>> try{
>> boolean sent=false;
>> if(node!=null){
>> Message message=node.getMessage();
>> if(message!=null&&node.getRegionDestination()!=null){
>> DeadLetterStrategy deadLetterStrategy=node
>> .getRegionDestination().getDeadLetterStrategy();
>> if(deadLetterStrategy!=null){
>> if(deadLetterStrategy.isSendToDeadLetterQueue(message)){
>> long expiration=message.getExpiration();
>> message.setExpiration(0);
>> message.setProperty("originalExpiration",new Long(
>> expiration));
>> if(!message.isPersistent()){
>> message.setPersistent(true);
>> message.setProperty("originalDeliveryMode",
>> "NON_PERSISTENT");
>> }
>> // The original destination and transaction id do
>> // not get filled when the message is first
>> // sent,
>> // it is only populated if the message is routed to
>> // another destination like the DLQ
>> ActiveMQDestination deadLetterDestination=deadLetterStrategy
>> .getDeadLetterQueueFor(message
>> .getDestination());
>> if (context.getBroker()==null) {
>> context.setBroker(getRoot());
>> }
>> BrokerSupport.resend(context,message,
>> deadLetterDestination);
>> sent=true;
>> }
>> }
>> }
>> }
>> if(sent==false){
>> LOG.warn("Failed to send "+node+" to dead letter queue");
>> }
>> }catch(Exception e){
>> LOG.warn("Failed to pass expired message to dead letter queue",e);
>> }
>> }
>>
>> To be more specific, the last "if":
>>
>> if(sent==false){
>> LOG.warn("Failed to send "+node+" to dead letter queue");
>> }
>>
>>
>> Unfortunately, there is neither a java-doc nor a possibility to
>> increase
>> the logging-output, so i am clueless why "sent=false" is true.
>>
>> If you look at the source-code above, there are four possibilites:
>>
>> node=null || message=null || node.getRegionDestination()=null ||
>> deadLetterStrategy=null
>>
>> And that's where i am stuck....
>>
>> I would really appreciate it, if some of you guys could help me
>> out......
>>
>
> --
> View this message in context: http://www.nabble.com/Failed-to-send-.....-to-dead-letter-queue-tp16606547s2354p16609061.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Failed to send ..... to dead letter queue
Posted by j0llyr0g3r <ti...@wincor-nixdorf.com>.
Update again:
i have to correct myself: This is NOT an AMQ / Java problem, but a problem
with SLES9. (All of my testservers run on sles9, that's why i thought it
would be an AMQ / Java problem)
I can reproduce above described behaviour:
Producer on Ubuntu / Java 1.6 -> Broker on Sles 9 / Java 1.6 -> Consumer
on Sles 9 / Java 1.6
=> NOT working!
If i substitute "SLES 9" with any other distribution (tested so far: Ubuntu
and gentoo) + java 1.6
=> working!
I really don't know what the hell SLES 9 is doing, and i don't care, because
the first thing i will do is to remove this shitty product immediately from
all of my testservers. (Not the first time SLES 9 is a pain in the ass in
comparison to other distributions)
j0llyr0g3r wrote:
>
> Hey folks,
>
> i really hope someone can help me out, since i am running out of ideas....
>
> My problem seems quite simple:
>
> -> I have a self-written message producer using AMQ-libraries
> -> I have a self-written message consumer using AMQ-libraries
> -> And a (single) AMQ-Broker in the middle
> -> All used AMQ-libraries have the some version (version 5, downloaded one
> week ago)
> -> consumer and producer use "real" tcp-connections, _not_
> "in-jvm"-connections, even when they are running on the same machine
>
> Now, if all of the three components (producer + broker + consumer) run on
> the same host, everything works _fine_....
>
> The problem:
>
> If all 3 components run on different hosts, nothing happens, meaning:
>
> -> My producer "hangs":
>
> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,085
> Sending message.....
> INFO (SendSyncCmd.send) 2008-04-08 11:18:11,090
> Waiting for reponse message...
>
> My producer sends to the queue:
>
> Publishing a Message with size 255 to queue: CMD.ESF-PROTO-2
>
> -> My consumer prints nothing out, i.e. receives nothing.
>
> But he listens on right queue: CMD.ESF-PROTO-2
>
> -> Now to the interesting part: The broker.
>
> If i take a look at the web-interface i see something like:
>
> Name Number Of Pending Messages Number Of Consumers
>
> CMD.ESF-PROTO-2 -1 1
>
> Number Of Pending Messages = -1? What is this supposed to mean?
>
> If i look at the command-line output of the broker i see:
>
> WARN RegionBroker - Failed to send ActiveMQBytesMessage
> {commandId = 7, responseRequired = false, messageId =
> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1:1,
> originalDestination = null, originalTransactionId = null, producerId =
> ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1:1, destination =
> queue://CMD.ESF-PROTO-2, transactionId = null, expiration = 1207822630933,
> timestamp = 1207822630678, arrival = 0, brokerInTime = 1209266957622,
> brokerOutTime = 0, correlationId = null, replyTo =
> temp-queue://ID:sdoesmon.wincor-nixdorf.com-41873-1207822630035-0:0:1,
> persistent = false, type = null, priority = 4, groupID = null,
> groupSequence = 0, targetConsumerId = null, compressed = false, userID =
> null, content = org.apache.activemq.util.ByteSequence@ee3dee,
> marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
> size = 0, properties = null, readOnlyProperties = false, readOnlyBody =
> false, droppable = false} ActiveMQBytesMessage{ bytesOut = null, dataOut =
> null, dataIn = null } to dead letter queue
>
> So, in a nutshell, the broker tells me:
>
> Failed to send ActiveMQBytesMessage ......to dead letter queue
>
> But tells in the same output:
>
> destination = queue://CMD.ESF-PROTO-2
>
> which is correct! This queue exists and i got a consumer listening!
>
> Now, if i look in the source-code of activemq, there is only one location
> where this message is generated:
>
> -> package org.apache.activemq.broker.region;
> -> Class RegionBroker
> -> Method sendToDeadLetterQueue()
>
> public void sendToDeadLetterQueue(ConnectionContext context,
> MessageReference node){
> try{
> boolean sent=false;
> if(node!=null){
> Message message=node.getMessage();
> if(message!=null&&node.getRegionDestination()!=null){
> DeadLetterStrategy deadLetterStrategy=node
> .getRegionDestination().getDeadLetterStrategy();
> if(deadLetterStrategy!=null){
> if(deadLetterStrategy.isSendToDeadLetterQueue(message)){
> long expiration=message.getExpiration();
> message.setExpiration(0);
> message.setProperty("originalExpiration",new Long(
> expiration));
> if(!message.isPersistent()){
> message.setPersistent(true);
> message.setProperty("originalDeliveryMode",
> "NON_PERSISTENT");
> }
> // The original destination and transaction id do
> // not get filled when the message is first
> // sent,
> // it is only populated if the message is routed to
> // another destination like the DLQ
> ActiveMQDestination deadLetterDestination=deadLetterStrategy
> .getDeadLetterQueueFor(message
> .getDestination());
> if (context.getBroker()==null) {
> context.setBroker(getRoot());
> }
> BrokerSupport.resend(context,message,
> deadLetterDestination);
> sent=true;
> }
> }
> }
> }
> if(sent==false){
> LOG.warn("Failed to send "+node+" to dead letter queue");
> }
> }catch(Exception e){
> LOG.warn("Failed to pass expired message to dead letter queue",e);
> }
> }
>
> To be more specific, the last "if":
>
> if(sent==false){
> LOG.warn("Failed to send "+node+" to dead letter queue");
> }
>
>
> Unfortunately, there is neither a java-doc nor a possibility to increase
> the logging-output, so i am clueless why "sent=false" is true.
>
> If you look at the source-code above, there are four possibilites:
>
> node=null || message=null || node.getRegionDestination()=null ||
> deadLetterStrategy=null
>
> And that's where i am stuck....
>
> I would really appreciate it, if some of you guys could help me out......
>
--
View this message in context: http://www.nabble.com/Failed-to-send-.....-to-dead-letter-queue-tp16606547s2354p16609061.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.