You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2020/10/11 14:54:11 UTC
[camel] branch master updated (e43e682 -> bb580c0)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git.
from e43e682 Sync Properties
new d7c437e Changes S3Object manipulation to avoid consume large S3 into the memory and consume it as stream instead.
new 672ebdc Adds integration tests for CAMEL-14929. Adjusts documentation.
new bb580c0 Regen and fixed CS
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../camel/component/aws2/s3/AWS2S3Endpoint.java | 14 +--
.../s3/integration/S3ConsumerIntegrationTest.java | 117 +++++++++++++++++++++
2 files changed, 125 insertions(+), 6 deletions(-)
[camel] 02/03: Adds integration tests for CAMEL-14929. Adjusts
documentation.
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 672ebdc3e0098554292bc394a45c8c54f4a57e9a
Author: Anderson Vaz <de...@andersonvaz.com>
AuthorDate: Sat Oct 10 18:16:01 2020 +0200
Adds integration tests for CAMEL-14929.
Adjusts documentation.
---
.../camel/catalog/docs/aws2-s3-component.adoc | 4 +-
.../src/main/docs/aws2-s3-component.adoc | 4 +-
.../s3/integration/S3ConsumerIntegrationTest.java | 116 +++++++++++++++++++++
.../modules/ROOT/pages/aws2-s3-component.adoc | 4 +-
4 files changed, 122 insertions(+), 6 deletions(-)
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
index 1c2525b..f537122 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
+| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
index 1c2525b..f537122 100644
--- a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
+++ b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
+| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
index 284767e..f301258 100644
--- a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
+++ b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
@@ -16,6 +16,13 @@
*/
package org.apache.camel.component.aws2.s3.integration;
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.stream.Collectors;
+
import org.apache.camel.BindToRegistry;
import org.apache.camel.EndpointInject;
import org.apache.camel.Exchange;
@@ -26,11 +33,19 @@ import org.apache.camel.component.aws2.s3.AWS2S3Constants;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.junit5.CamelTestSupport;
import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.ResponseInputStream;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
+import software.amazon.awssdk.services.s3.model.GetObjectResponse;
+
+import static org.hamcrest.CoreMatchers.*;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertThrows;
@Disabled("Must be manually tested. Provide your own accessKey and secretKey!")
public class S3ConsumerIntegrationTest extends CamelTestSupport {
@@ -83,11 +98,112 @@ public class S3ConsumerIntegrationTest extends CamelTestSupport {
assertMockEndpointsSatisfied();
}
+ @Test
+ @DisplayName("Should consume S3StreamObject when include body is true and should close the stream when autocloseBody is true")
+ public void shouldConsumeS3StreamObjectWhenIncludeBodyIsTrueAndNotCloseStreamWhenAutoCloseBodyIsTrue()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyTrueAutoCloseTrue");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyTrueAutoCloseTrue", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(String.class)));
+ assertThat(exchange.getIn().getBody(String.class), is("Test"));
+ }
+
+ @Test
+ @DisplayName("Should not consume S3StreamObject when include body is false and should not close the stream when autocloseBody is false")
+ public void shouldNotConsumeS3StreamObjectWhenIncludeBodyIsFalseAndNotCloseStreamWhenAutoCloseBodyIsFalse()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyFalseAutoCloseFalse");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyFalseAutoCloseFalse", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(ResponseInputStream.class)));
+ assertDoesNotThrow(() -> {
+ final ResponseInputStream<GetObjectResponse> inputStream = exchange.getIn().getBody(ResponseInputStream.class);
+ try (BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) {
+ final String text = reader.lines().collect(Collectors.joining());
+ assertThat(text, is("Test"));
+ }
+ });
+ }
+
+ @Test
+ @DisplayName("Should not consume S3StreamObject when include body is false and should close the stream when autocloseBody is true")
+ public void shouldNotConsumeS3StreamObjectWhenIncludeBodyIsFalseAndCloseStreamWhenAutoCloseBodyIsTrue()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyFalseAutoCloseTrue");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyFalseAutoCloseTrue", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(ResponseInputStream.class)));
+ assertThrows(IOException.class, () -> {
+ final ResponseInputStream<GetObjectResponse> inputStream = exchange.getIn().getBody(ResponseInputStream.class);
+ inputStream.read();
+ });
+ }
+
@Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
@Override
public void configure() throws Exception {
+ String template = "aws2-s3://mycamel?autoCreateBucket=true&includeBody=%s&autocloseBody=%s";
+ String includeBodyTrueAutoCloseTrue = String.format(template, true, true);
+ String includeBodyFalseAutoCloseFalse = String.format(template, false, false);
+ String includeBodyFalseAutoCloseTrue = String.format(template, false, true);
+ from("direct:includeBodyTrueAutoCloseTrue").pollEnrich(includeBodyTrueAutoCloseTrue, 5000).to("mock:result");
+ from("direct:includeBodyFalseAutoCloseFalse").pollEnrich(includeBodyFalseAutoCloseFalse, 5000).to("mock:result");
+ from("direct:includeBodyFalseAutoCloseTrue").pollEnrich(includeBodyFalseAutoCloseTrue, 5000).to("mock:result");
+
String awsEndpoint = "aws2-s3://mycamel?autoCreateBucket=false";
from("direct:putObject").startupOrder(1).to(awsEndpoint).to("mock:result");
diff --git a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
index da54118..203b3b4 100644
--- a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
+++ b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
@@ -80,11 +80,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
[camel] 01/03: Changes S3Object manipulation to avoid consume large
S3 into the memory and consume it as stream instead.
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit d7c437eb3f476e366787bac80a384bed17f463fe
Author: Anderson Vaz <de...@andersonvaz.com>
AuthorDate: Tue Sep 22 09:27:34 2020 +0200
Changes S3Object manipulation to avoid consume large S3 into the memory and consume it as stream instead.
---
.../org/apache/camel/catalog/docs/aws2-s3-component.adoc | 4 ++--
.../camel-aws2-s3/src/main/docs/aws2-s3-component.adoc | 4 ++--
.../org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java | 14 ++++++++------
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
index f537122..1c2525b 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
index f537122..1c2525b 100644
--- a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
+++ b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java b/components/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java
index c9e3b9c..03b47e5 100644
--- a/components/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java
+++ b/components/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java
@@ -175,7 +175,7 @@ public class AWS2S3Endpoint extends ScheduledPollEndpoint {
throw new RuntimeCamelException(e);
}
} else {
- message.setBody(null);
+ message.setBody(s3Object);
}
message.setHeader(AWS2S3Constants.KEY, key);
@@ -193,12 +193,14 @@ public class AWS2S3Endpoint extends ScheduledPollEndpoint {
message.setHeader(AWS2S3Constants.REPLICATION_STATUS, s3Object.response().replicationStatus());
message.setHeader(AWS2S3Constants.STORAGE_CLASS, s3Object.response().storageClass());
- /**
- * If includeBody != true, it is safe to close the object here. If includeBody == true, the caller is
- * responsible for closing the stream and object once the body has been fully consumed. As of 2.17, the consumer
- * does not close the stream or object on commit.
+ /*
+ * If includeBody == true, it is safe to close the object here because the S3Object
+ * was consumed already. If includeBody != true, the caller is responsible for
+ * closing the stream once the body has been fully consumed or use the autoCloseBody
+ * configuration to automatically schedule the body closing at the end of exchange.
+ * TODO REVIEW THIS STATEMENT As of 2.17, the consumer does not close the stream or object on commit.
*/
- if (!configuration.isIncludeBody()) {
+ if (configuration.isIncludeBody()) {
IOHelper.close(s3Object);
} else {
if (configuration.isAutocloseBody()) {
[camel] 03/03: Regen and fixed CS
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit bb580c056b8f2b6d5b716f9b905a5eeff4bf4bdc
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Sun Oct 11 15:34:38 2020 +0200
Regen and fixed CS
---
.../component/aws2/s3/integration/S3ConsumerIntegrationTest.java | 3 ++-
docs/components/modules/ROOT/pages/aws2-s3-component.adoc | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
index f301258..4ba37f0 100644
--- a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
+++ b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
@@ -201,7 +201,8 @@ public class S3ConsumerIntegrationTest extends CamelTestSupport {
String includeBodyFalseAutoCloseFalse = String.format(template, false, false);
String includeBodyFalseAutoCloseTrue = String.format(template, false, true);
from("direct:includeBodyTrueAutoCloseTrue").pollEnrich(includeBodyTrueAutoCloseTrue, 5000).to("mock:result");
- from("direct:includeBodyFalseAutoCloseFalse").pollEnrich(includeBodyFalseAutoCloseFalse, 5000).to("mock:result");
+ from("direct:includeBodyFalseAutoCloseFalse").pollEnrich(includeBodyFalseAutoCloseFalse, 5000)
+ .to("mock:result");
from("direct:includeBodyFalseAutoCloseTrue").pollEnrich(includeBodyFalseAutoCloseTrue, 5000).to("mock:result");
String awsEndpoint = "aws2-s3://mycamel?autoCreateBucket=false";
diff --git a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
index 203b3b4..da54118 100644
--- a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
+++ b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
@@ -80,11 +80,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
+| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]