Missing producer state error when using FailoverTransport in ActiveMQ 5.10.0

[Osman Rafiq <or...@cfhcloud.com>]

Hi guys,

We have a single broker test setup that is used by clients to retrieve data
from servers using a request-response pattern i.e.;
1) Client creates TempQueue
2) Client publishes messages on Topic with JMSReplyTo = TempQueue
3) Server consumes messages
4) Server creates dedicated message publisher for TempQueue specified in
JMSReplyTo
5) Server publishes reply to TempQueue

The server is connected to the broker using the FailoverTransport over TCP;
failover:(tcp://
10.10.249.1:61616?trace=true&dynamicManagement=true&wireFormat.tightEncodingEnabled=false&wireFormat.stackTraceEnabled=false&wireFormat.cacheEnabled=false&wireFormat.tcpNoDelayEnabled=true&socket.tcpNoDelay=true)?jms.copyMessageOnSend=false

The setup works quite well and survives broker restarts.
However we recently observed the following error after the broker machine
unexpectedly crashed;

4968 2016-01-21 23:12:18,016 | WARN  | Send failed for:
ActiveMQBytesMessage {commandId = 215860, responseRequired = false,
messageId = ID:OMS-D2-R1-50893-1452946777125-21:1:1:12:30216,
originalDestination = null, originalTransactionId = null, producerId =
ID:TS2OMS-D2-R1-50893-1452946777125-21:1:1:12, destination =
temp-queue://ID:FIX-D2-R1-49194-635885175520704255-0:10:1, transactionId =
null, expiration = 0, timestamp = 1453417937941, arrival = 0, brokerInTime
= 1453417938005, brokerOutTime = 0, correlationId = null, replyTo = null,
persistent = false, type = null, priority = 4, groupID = null,
groupSequence = 0, targetConsumerId = null, compressed = false, userID =
null, content = org.apache.activemq.util.ByteSequence@45dcc4fd,
marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
size = 0, properties = null, readOnlyProperties = false, readOnlyBody =
false, droppable = false, jmsXGroupFirstForConsumer = false}
ActiveMQBytesMessage{ bytesOut = null, dataOut = null, dataIn = null },
missing producer state for:
org.apache.activemq.broker.ProducerBrokerExchange@6dd1a9d1 |
org.apache.activemq.broker.region.Queue | ActiveMQ Transport:
tcp:///10.10.1.2:49411@61616
4969 2016-01-21 23:12:18,017 | WARN  | Async error occurred:  |
org.apache.activemq.broker.TransportConnection.Service | ActiveMQ
Transport: tcp:///10.10.1.2:49411@61616
4971        at org.apache.activemq.broker.region.Queue.send(Queue.java:621)
4972        at
org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:132)
4973        at
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:419)
4974        at
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:468)
4975        at
org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:297)
4976        at
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:152)
4977        at
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
4978        at
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:307)
4979        at
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:152)
4981        at
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:152)
4982        at
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:157)
4983        at
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:541)
4985        at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334)
4986        at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)

The error persists until the either the server or the client is restarted.

I am really hoping you can point me in the right direction with this one, I
have been staring at the ActiveMQ source code for days now and cant really
see how the producer state could have gone missing when using the
FailoverTransport.

Broker version: ActiveMQ 5.11.1
Server version: ActiveMQ 5.10.0
Client version: Apache NMS 1.6.0

Best regards
Osman

-- 
*This e-mail and any attachment may contain confidential and/or privileged 
information. If you are not the intended recipient (or have received this 
e-mail in error) please notify the sender immediately and delete this 
message and any attachment from your system. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.*