You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "MykhailoVlakh (JIRA)" <ji...@apache.org> on 2019/04/18 10:21:00 UTC
[jira] [Created] (CAMEL-13433) S3: Exchange body stream is loaded
into memory to calculate content length which is already set via headers
MykhailoVlakh created CAMEL-13433:
-------------------------------------
Summary: S3: Exchange body stream is loaded into memory to calculate content length which is already set via headers
Key: CAMEL-13433
URL: https://issues.apache.org/jira/browse/CAMEL-13433
Project: Camel
Issue Type: Bug
Components: camel-aws
Affects Versions: 3.0.0-M1, 2.22.1, 2.23.2
Reporter: MykhailoVlakh
My code uploads large files to S3 storage (500 MB and bigger). I create an exchange with properly set meta information headers including the _CamelAwsS3ContentLength_, as a body I provide a stream. Everything worked well in Camel 2.14.3 but when I upgraded to Camel ti version 2.22.1 I suddenly began getting OOM issues from time to time. I did some debugging and found that the issue is in S3Producer that loads content of the body stream into memory.
{code:java}
public void processSingleOp(final Exchange exchange) throws Exception {
ObjectMetadata objectMetadata = determineMetadata(exchange); <== here we already set length
File filePayload = null;
InputStream is = null;
ByteArrayOutputStream baos = null;
Object obj = exchange.getIn().getMandatoryBody();
PutObjectRequest putObjectRequest = null;
// Need to check if the message body is WrappedFile
if (obj instanceof WrappedFile) {
obj = ((WrappedFile<?>)obj).getFile();
}
if (obj instanceof File) {
filePayload = (File)obj;
is = new FileInputStream(filePayload);
} else {
is = exchange.getIn().getMandatoryBody(InputStream.class);
baos = determineLengthInputStream(is); <====== the issue is here
objectMetadata.setContentLength(baos.size());
is = new ByteArrayInputStream(baos.toByteArray());
}
...
{code}
This is a defect since, as I mentioned, I already provide content length via headers and in the first line of the method populates that information into object metadata.
I think there should be some if check that skips content length size calculation if it is already known.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)