You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Luca Zenti (Created) (JIRA)" <ji...@apache.org> on 2011/11/14 20:06:51 UTC
[jira] [Created] (AMQ-3593) Messages with empty
marshalledProperties
Messages with empty marshalledProperties
----------------------------------------
Key: AMQ-3593
URL: https://issues.apache.org/jira/browse/AMQ-3593
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.5.1, 5.4.2
Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
* JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
Reporter: Luca Zenti
Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
{noformat}
import java.io.DataOutputStream;
import java.io.IOException;
import javax.jms.Session;
import org.apache.activemq.ActiveMQSession;
import org.apache.activemq.command.ActiveMQBytesMessage;
import org.apache.activemq.command.Message;
import org.apache.activemq.util.ByteArrayOutputStream;
import org.apache.activemq.util.MarshallingSupport;
public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
{
private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
public EarlySerializingBytesMessage(Session session)
{
ActiveMQSession activeMqSession = (ActiveMQSession)session;
setConnection(activeMqSession.getConnection());
}
/*
* We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
* arriving empty on the client side.
* This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
* the client.
* This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
* To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
* the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
* The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
* sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
* as soon as we try to read them.
* Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
* then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
* queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
* method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
*/
@Override
public Message copy()
{
try
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream os = new DataOutputStream(baos);
MarshallingSupport.marshalPrimitiveMap(properties, os);
os.close();
marshalledProperties = baos.toByteSequence();
}
catch(IOException exc)
{
logger.error("Error marhalling message properties, message was " + this, exc);
}
return super.copy();
}
}
{noformat}
The problem happens both on Java and C# clients, with the following stacktraces:
Java:
{noformat}
javax.jms.JMSException: java.io.EOFException
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
at org.apache.activemq.command.Message.getProperty(Message.java:159)
at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
... 11 more
{noformat}
C#:
{noformat}
System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
at System.IO.MemoryStream.InternalReadInt32()
at System.IO.BinaryReader.ReadInt32()
at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
...
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3593) Messages with empty
marshalledProperties
Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151550#comment-13151550 ]
Timothy Bish commented on AMQ-3593:
-----------------------------------
Recommend you validate this against a 5.6 SNAPSHOT to see if your issue has already been fixed.
> Messages with empty marshalledProperties
> ----------------------------------------
>
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
>
>
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
> This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
> We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
> I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
> Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
> The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
> I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
> I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
> My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
> This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
> Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
> {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
>
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
>
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
> * sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
> * queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
>
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
> at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
> at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
> at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
> at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
> at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3593) Messages with empty
marshalledProperties
Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155349#comment-13155349 ]
Timothy Bish commented on AMQ-3593:
-----------------------------------
A JUnit test case that can be included in the unit tests would help in terms of investigation and in ensuring this stays fixed in the future.
> Messages with empty marshalledProperties
> ----------------------------------------
>
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
>
>
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
> This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
> We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
> I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
> Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
> The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
> I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
> I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
> My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
> This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
> Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
> {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
>
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
>
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
> * sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
> * queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
>
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
> at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
> at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
> at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
> at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
> at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3593) Messages with empty
marshalledProperties
Posted by "Luca Zenti (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155756#comment-13155756 ]
Luca Zenti commented on AMQ-3593:
---------------------------------
I tried to isolate the problem and to replicate it in a "predictable" way, but I did not manage to. I believe it is something related to concurrency and to how the thread scheduler behaves on Linux, but I couldn't isolate the case.
Also, as far as I could understand, properties are serialized separately for each topic subscriber when the message is finally sent to the network. As far as the various clients are configured to use the same serialized format (does it make any difference for the properties?), wouldn't it be better to do it just once? Obviously, it would increase the memory footprint of each message, but since they are copied into new instances to dispatch to each subscriber queue, it would be possible to just copy the serialized version, potentially saving some memory as the properties are quite often primitive types and the serialized byte array could be the same object for all message instances.
I guess the same is applicable for the message's body, in my specific case it probably doesn't make much difference because we mainly use bytes messages, but especially with object messages it could save a lot of work to the broker.
I know that conditions are quite specific, though.
> Messages with empty marshalledProperties
> ----------------------------------------
>
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
>
>
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
> This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
> We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
> I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
> Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
> The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
> I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
> I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
> My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
> This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
> Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
> {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
>
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
>
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
> * sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
> * queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
>
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
> at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
> at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
> at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
> at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
> at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AMQ-3593) Messages with empty
marshalledProperties
Posted by "Luca Zenti (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luca Zenti updated AMQ-3593:
----------------------------
Attachment: Subscriber.java
PublisherAndBroker.java
> Messages with empty marshalledProperties
> ----------------------------------------
>
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
>
>
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
> This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
> We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
> I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
> Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
> The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
> I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
> I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
> My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
> This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
> Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
> {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
>
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
>
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
> * sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
> * queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
>
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
> at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
> at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
> at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
> at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
> at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3593) Messages with empty
marshalledProperties
Posted by "Luca Zenti (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155301#comment-13155301 ]
Luca Zenti commented on AMQ-3593:
---------------------------------
I've tested it against activemq-core-5.6-SNAPSHOT.jar dated Sat Nov 19 03:23:47 and it still happens.
> Messages with empty marshalledProperties
> ----------------------------------------
>
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
>
>
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. This causes an exception trying to unmarshall them.
> This happens very rarely, but we've got at least 2-3 cases a day per subscriber in our production environment.
> We use a broker embedded with a server component that publishes messages on topics. The subscribers are remote.
> I was unable to replicate the problem on my Windows workstation, nor to reproduce it precisely.
> Attached you can find a simple pair of programs (the publisher and the subscriber) that can reproduce the issue within 5 minutes at most on my environment. This happens only if at least one of the subscribers runs on a remote machine and it seems to happen quickly with more clients connected.
> The publishing program has a command line arguments that allows to adjust the publishing rate, in my case with 10,000 messages per second and 2 clients the problem happens within a couple of minutes.
> I also found a workaround for this problem: as far as I can understand, my original message is copied as a Java object to the broker (the broker is embedded, so there is no need to serialize it), then it is copied using its copy method to one or more new ActiveMQMessage instances and these are pushed into the consumer queues and then serialized to be sent over tcp.
> I think that under some particular conditions the properties are not serialized into the marshalledProperties field or this happens after the message has been actually sent.
> My workaround is to force the marshalling of properties into the marshalledProperties field when the message is copied using the copy method.
> This is not very clean, but still acceptable. The problem is that, in order for this to work, I need to create messages of a subclass of ActiveMqBytesMessage using an explicit constructor call rather that session.createBytesMessage() and I need to set the connection into them.
> Here is the code of my redefined class (this is the BytesMessage version, but the same is applicable to other kinds of messages):
> {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class);
>
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
>
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and avoid a problem we've found with serialized properties
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it serializes every message just before
> * sending it and it happens that the marshalled properties are missing the resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, in this case) and pushed into the subscribers'
> * queues (a topic is actually implemented as a series of queues, one for each subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
>
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
> at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
> at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
> at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
> at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
> at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira