[jira] [Created] (AMQ-8351) MaxFrameSize is not protecting against allocating big buffer

["Marcin (Jira)" <ji...@apache.org>]

Marcin created AMQ-8351:
---------------------------

             Summary: MaxFrameSize is not protecting against allocating big buffer
                 Key: AMQ-8351
                 URL: https://issues.apache.org/jira/browse/AMQ-8351
             Project: ActiveMQ
          Issue Type: Bug
          Components: Transport
    Affects Versions: 5.16.2, 5.15.14, 5.14.5, 5.13.5, 5.12.2
            Reporter: Marcin


I found that some incoming data can allocate much bigger buffer (up to max int size) than frame size limit. This can lead to oom. I created junit test to show the problem:

 

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInput;
import java.io.DataInputStream;
import java.io.IOException;

import com.google.common.primitives.Ints;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

class OpenWireFormatTest {

 @Test
 void maxFrameSizeTest() {
 ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
 try {
 outputStream.write(Ints.toByteArray(999));
 outputStream.write(1);
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(1);
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 outputStream.write(Ints.toByteArray(2000000000));
 } catch (IOException e) {
 e.printStackTrace();
 }



 ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(outputStream.toByteArray());
 DataInput dataInput = new DataInputStream(byteArrayInputStream);

 OpenWireFormat openWireFormat = new OpenWireFormat();
 openWireFormat.setMaxFrameSize(1000);

 final IOException ioException = assertThrows(IOException.class, () -> openWireFormat.unmarshal(dataInput));
 assertEquals("Frame size of 1907 MB larger than max allowed 100 MB", ioException.getMessage());
 }



--
This message was sent by Atlassian Jira
(v8.3.4#803005)