You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "ruffp (JIRA)" <ji...@apache.org> on 2017/07/10 12:09:00 UTC
[jira] [Commented] (AMQ-6596) Out Of Memory error reported on
ActiveMQ client during openwire unmarshalling
[ https://issues.apache.org/jira/browse/AMQ-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080233#comment-16080233 ]
ruffp commented on AMQ-6596:
----------------------------
We noticed the same behavior by sending arbitrary data to the openwire port in our ActiveMQ 5.10.2.
Not sure if this is related to the same bug, but executing the following bash script on the same machine:
{code}
#!/bin/bash
if [ -t 0 ]; then stty -echo -icanon -icrnl time 0 min 0; fi
count=0
keypress=''
while [ "x$keypress" = "x" ]; do
let count+=1
echo -ne $count'\r'
keypress="`cat -v`"
dd if=/dev/urandom bs=10000 count=120 > /dev/tcp/localhost/61616
done
if [ -t 0 ]; then stty sane; fi
echo "You pressed '$keypress' after $count loop iterations"
exit 0
{code}
makes the OutOfMemoryError happens few times even the data are smaller (try with bs=100 instead).
By executing this script on the latest ActiveMQ 5.14.5 I get the same results.
Here's a extract of the logs:
{noformat}
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55550 failed: java.io.IOException: Unknown data type: -11
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55551 failed: java.io.IOException: Frame size of 1611 MB larger than max allowed 100 MB
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55553 failed: java.io.IOException: Unknown data type: -46
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55552 failed: java.io.IOException: Frame size of 361 MB larger than max allowed 100 MB
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55555 failed: java.io.IOException: Frame size of 1271 MB larger than max allowed 100 MB
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55554 failed: java.io.UTFDataFormatException: malformed input around byte 0
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55557 failed: java.io.IOException: Unexpected error occured: java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:53624 failed: java.io.IOException: Frame size of 467 MB larger than max allowed 100 MB
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55556 failed: java.io.IOException: Frame size of 1382 MB larger than max allowed 100 MB
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55548 failed: java.io.EOFException
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55543 failed: java.io.IOException: Unknown data type: -110
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55539 failed: java.io.IOException: Unknown data type: -62
INFO | jvm 1 | 2017/07/10 10:27:30 | WARN | Transport Connection to: tcp://127.0.0.1:55542 failed: java.io.IOException: Unknown data type: -24
{noformat}
When our vulnerabiliy scanner (I don't know which one it was) created such requests, our ActiveMQ has blocked unexpectedly.
> Out Of Memory error reported on ActiveMQ client during openwire unmarshalling
> -----------------------------------------------------------------------------
>
> Key: AMQ-6596
> URL: https://issues.apache.org/jira/browse/AMQ-6596
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker, JMS client, Transport
> Affects Versions: 5.13.0, 5.13.5, 5.14.3
> Environment: SUSE Linux Enterprise Server 11 (x86_64), VERSION = 11, PATCHLEVEL = 3. Java Runtime: IBM Corporation 1.7.0
> Reporter: Konstantinos Pistopoulos
> Priority: Critical
> Attachments: packet.raw
>
>
> ActiveMQ crashes during a vulnerability scanning using Qualys vulnerability scanner.
> {code}
> 2017-02-10 14:30:18,631 [0.1:55345@61616] WARN Transport - Transport Connection to: tcp://127.0.0.1:55345 failed: java.io.IOException: Unexpected error occurred: java.lang.OutOfMemoryError: Java heap space
> {code}
> This OOM error is caused due to an attempt to initialize a huge byte array of 2131230728 bytes size. The problem still occurs even if we define the maxFrameSize to 100MB.
> It seems that the first comparison with the maxFrameSize is successful (method unmarshal) but in the later evaluation of dataIn.readInt() in method looseUnmarshalByteSequence, a huge value is evaluated (2131230728 bytes)
> {code}
> @Override
> public Object unmarshal(DataInput dis) throws IOException {
> DataInput dataIn = dis;
> if (!sizePrefixDisabled) {
> int size = dis.readInt();
> if (size > maxFrameSize) {
> throw IOExceptionSupport.createFrameSizeException(size, maxFrameSize);
> }
> // int size = dis.readInt();
> // byte[] data = new byte[size];
> // dis.readFully(data);
> // bytesIn.restart(data);
> // dataIn = bytesIn;
> }
> return doUnmarshal(dataIn);
> }
> {code}
> {code}
> protected ByteSequence looseUnmarshalByteSequence(DataInput dataIn) throws IOException {
> ByteSequence rc = null;
> if (dataIn.readBoolean()) {
> int size = dataIn.readInt();
> byte[] t = new byte[size];
> dataIn.readFully(t);
> rc = new ByteSequence(t, 0, size);
> }
> return rc;
> }
> {code}
> Relevant parts of the thread dump can be found below:
> {code}
> WARNING : OutOfMemoryError possibly caused by 2131230728 bytes requested for object of class 081A5700 from memory space 'Flat' id=080B1898
> {code}
> {code}
> Thread Name
> ActiveMQ Transport: tcp:///10.4.240.10:55345@61616
> State
> Runnable
> Java Stack
> at org/apache/activemq/openwire/v12/BaseDataStreamMarshaller.looseUnmarshalByteSequence(BaseDataStreamMarshaller.java:638)
> at org/apache/activemq/openwire/v12/WireFormatInfoMarshaller.looseUnmarshal(WireFormatInfoMarshaller.java:132)
> at org/apache/activemq/openwire/OpenWireFormat.doUnmarshal(OpenWireFormat.java:367(Compiled Code))
> at org/apache/activemq/openwire/OpenWireFormat.unmarshal(OpenWireFormat.java:278(Compiled Code))
> at org/apache/activemq/transport/tcp/TcpTransport.readCommand(TcpTransport.java:240(Compiled Code))
> at org/apache/activemq/transport/tcp/TcpTransport.doRun(TcpTransport.java:232(Compiled Code))
> at org/apache/activemq/transport/tcp/TcpTransport.run(TcpTransport.java:215)
> at java/lang/Thread.run(Thread.java:863)
> {code}
> The definition of the transportConnector without the definition of the maxFrameSize is the following :
> {code}
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> </transportConnectors>
> {code}
> The definition of the transportConnector after the definition of the maxFrameSize :
> {code}
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616?wireFormat.maxFrameSize=104857600"/>
> </transportConnectors>
> {code}
> We have reproduced this with versions 5.13.0, 5.13.5 and 5.14.3 but this problem is probably related to other versions too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)