You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by khandelwalanuj <kh...@gmail.com> on 2015/01/13 15:05:27 UTC
ActiveMQ Stomp producer messages size limitation
Hi,
I am not able to send message more than 100MB to ActiveMQ broker. Producer
throws below exception:
/[20150113 03:16:59:609 stomp_client.py:80 ERROR] Received an error:
org.apache.activemq.transport.stomp.ProtocolException: The maximum data
length was exceeded
at
org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:128)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:745)/
I checked the source code
(http://activemq.apache.org/maven/apidocs/src-html/org/apache/activemq/transport/stomp/StompWireFormat.html)
and found that "private static final int MAX_DATA_LENGTH = 1024 * 1024 *
100;" is causing the issue. How can I change it ? what should I do to send
140 size messages ?
Thanks,
Anuj
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Stomp producer messages size limitation
Posted by Timothy Bish <ta...@gmail.com>.
On 01/13/2015 09:05 AM, khandelwalanuj wrote:
> Hi,
>
> I am not able to send message more than 100MB to ActiveMQ broker. Producer
> throws below exception:
>
> /[20150113 03:16:59:609 stomp_client.py:80 ERROR] Received an error:
> org.apache.activemq.transport.stomp.ProtocolException: The maximum data
> length was exceeded
> at
> org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:128)
> at
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
> at java.lang.Thread.run(Thread.java:745)/
>
> I checked the source code
> (http://activemq.apache.org/maven/apidocs/src-html/org/apache/activemq/transport/stomp/StompWireFormat.html)
> and found that "private static final int MAX_DATA_LENGTH = 1024 * 1024 *
> 100;" is causing the issue. How can I change it ? what should I do to send
> 140 size messages ?
>
> Thanks,
> Anuj
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
You need to configure a large maxFrameSize on the STOMP
transportConnector in your Broker configuration, see:
http://activemq.apache.org/xml-configuration.html
--
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: ActiveMQ Stomp producer messages size limitation
Posted by Tim Bain <tb...@alumni.duke.edu>.
1. As I said in my earlier email, you can download the code, change that
constant, and recompile, or you can split your messages and make your
clients handle that properly (which may be problematic since you need to
ensure that all parts of a given message are consumed, in order, by the
same consumer). Maybe someone else has a way to do this without doing one
of those, but I think you'd have heard it by now if someone had an answer
for you.
2. I can't answer that, but maybe someone else can.
3. Sounds like a great idea, and it sounds like it'll be an easy fix for
someone. (Maybe even you? You've clearly looked at this code enough to
know it better than most people.) Too bad it missed the cutoff for 5.11.0,
though.
Tim
On Wed, Feb 4, 2015 at 7:21 AM, khandelwalanuj <kh...@gmail.com>
wrote:
> Another concern here:
>
> As Tim mentioned :
> > You need to configure a large maxFrameSize on the STOMP
> transportConnector
> > in your Broker
> > configuration, see: http://activemq.apache.org/xml-configuration.html
>
> I looked into the source code of ActiveMQ. I don't think changing the
> 'maxFrameSize' will work here. From the stack trace I posted I can clearly
> see that 'MAX_DATA_LENGTH' and 'MAX_COMMAND_LENGTH' is exceeded when I try
> to send more than 100MB of data. And these are constants defined in
> StompWireFormat.java. I didn't find any way to modified these parameters in
> entire ActiveMQ source code.
>
> *My Questions:
> *
> 1. Is there any way I can send more than 100MB of data from STOMP (python
> and perl) producers?
> 2. What is the difference between 'MAX_DATA_LENGTH' and
> 'MAX_COMMAND_LENGTH'
> ?
> 3. These parameters should be configurable. Should I log a JIRA for this ?
>
> Thanks,
> Anuj
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847p4691050.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: ActiveMQ Stomp producer messages size limitation
Posted by khandelwalanuj <kh...@gmail.com>.
Another concern here:
As Tim mentioned :
> You need to configure a large maxFrameSize on the STOMP transportConnector
> in your Broker
> configuration, see: http://activemq.apache.org/xml-configuration.html
I looked into the source code of ActiveMQ. I don't think changing the
'maxFrameSize' will work here. From the stack trace I posted I can clearly
see that 'MAX_DATA_LENGTH' and 'MAX_COMMAND_LENGTH' is exceeded when I try
to send more than 100MB of data. And these are constants defined in
StompWireFormat.java. I didn't find any way to modified these parameters in
entire ActiveMQ source code.
*My Questions:
*
1. Is there any way I can send more than 100MB of data from STOMP (python
and perl) producers?
2. What is the difference between 'MAX_DATA_LENGTH' and 'MAX_COMMAND_LENGTH'
?
3. These parameters should be configurable. Should I log a JIRA for this ?
Thanks,
Anuj
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847p4691050.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Stomp producer messages size limitation
Posted by khandelwalanuj <kh...@gmail.com>.
Hi,
I am sending the complete stack trace, Please see below:
[I have added 'maxFrameSize' in broker URI, I didn't change anything on
client size: I have tried with and without 'maxframesize' but result is
always same. I am getting below error]:
*[20150204 19:00:54:306 stomp_client.py:80 ERROR] Received an error:
org.apache.activemq.transport.stomp.ProtocolException: The maximum data
length was exceeded*
at
org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:128)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:745)
[20150204 19:00:54:306 stomp_client.py:80 ERROR] Received an error:
*org.apache.activemq.transport.stomp.ProtocolException: The maximum command
length was exceeded
at
*org.apache.activemq.transport.stomp.StompWireFormat.readHeaderLine(StompWireFormat.java:158)
at
org.apache.activemq.transport.stomp.StompWireFormat.readLine(StompWireFormat.java:148)
at
org.apache.activemq.transport.stomp.StompWireFormat.parseAction(StompWireFormat.java:181)
at
org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:98)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:745)
Thanks,
Anuj
--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847p4691046.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ Stomp producer messages size limitation
Posted by Tim Bain <tb...@alumni.duke.edu>.
One option would be to download the 5.10.0 code, change the value of that
constant, build and deploy. But you'll have to test to make sure that
limit wasn't protecting the broker from some problem that happens with
messages larger than 100MB.
Another would be to split the message in half and have your consumer only
ack both messages when the last one is received. You have to make sure the
same consumer gets all the messages that comprise the original;
JMSCorrelationID might be useful here.
You could put the actual content somewhere both producer and consumer can
access and just pass a reference in the message. This might or might not
work for your use case, but if it does it'll allow you to reduce the memory
the broker needs, which is a good thing.
You could reduce the amount of data you're sending. What single message is
140MB in size, anyway?
On Jan 13, 2015 7:08 AM, "khandelwalanuj" <kh...@gmail.com>
wrote:
> Hi,
>
> I am not able to send message more than 100MB to ActiveMQ broker. Producer
> throws below exception:
>
> /[20150113 03:16:59:609 stomp_client.py:80 ERROR] Received an error:
> org.apache.activemq.transport.stomp.ProtocolException: The maximum data
> length was exceeded
> at
>
> org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:128)
> at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
> at java.lang.Thread.run(Thread.java:745)/
>
> I checked the source code
> (
> http://activemq.apache.org/maven/apidocs/src-html/org/apache/activemq/transport/stomp/StompWireFormat.html
> )
> and found that "private static final int MAX_DATA_LENGTH = 1024 * 1024 *
> 100;" is causing the issue. How can I change it ? what should I do to send
> 140 size messages ?
>
> Thanks,
> Anuj
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Stomp-producer-messages-size-limitation-tp4689847.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>