You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2020/10/11 14:54:13 UTC
[camel] 02/03: Adds integration tests for CAMEL-14929. Adjusts
documentation.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 672ebdc3e0098554292bc394a45c8c54f4a57e9a
Author: Anderson Vaz <de...@andersonvaz.com>
AuthorDate: Sat Oct 10 18:16:01 2020 +0200
Adds integration tests for CAMEL-14929.
Adjusts documentation.
---
.../camel/catalog/docs/aws2-s3-component.adoc | 4 +-
.../src/main/docs/aws2-s3-component.adoc | 4 +-
.../s3/integration/S3ConsumerIntegrationTest.java | 116 +++++++++++++++++++++
.../modules/ROOT/pages/aws2-s3-component.adoc | 4 +-
4 files changed, 122 insertions(+), 6 deletions(-)
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
index 1c2525b..f537122 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
+| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
index 1c2525b..f537122 100644
--- a/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
+++ b/components/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
@@ -78,11 +78,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
+| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
diff --git a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
index 284767e..f301258 100644
--- a/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
+++ b/components/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java
@@ -16,6 +16,13 @@
*/
package org.apache.camel.component.aws2.s3.integration;
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.stream.Collectors;
+
import org.apache.camel.BindToRegistry;
import org.apache.camel.EndpointInject;
import org.apache.camel.Exchange;
@@ -26,11 +33,19 @@ import org.apache.camel.component.aws2.s3.AWS2S3Constants;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.junit5.CamelTestSupport;
import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.ResponseInputStream;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
+import software.amazon.awssdk.services.s3.model.GetObjectResponse;
+
+import static org.hamcrest.CoreMatchers.*;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertThrows;
@Disabled("Must be manually tested. Provide your own accessKey and secretKey!")
public class S3ConsumerIntegrationTest extends CamelTestSupport {
@@ -83,11 +98,112 @@ public class S3ConsumerIntegrationTest extends CamelTestSupport {
assertMockEndpointsSatisfied();
}
+ @Test
+ @DisplayName("Should consume S3StreamObject when include body is true and should close the stream when autocloseBody is true")
+ public void shouldConsumeS3StreamObjectWhenIncludeBodyIsTrueAndNotCloseStreamWhenAutoCloseBodyIsTrue()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyTrueAutoCloseTrue");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyTrueAutoCloseTrue", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(String.class)));
+ assertThat(exchange.getIn().getBody(String.class), is("Test"));
+ }
+
+ @Test
+ @DisplayName("Should not consume S3StreamObject when include body is false and should not close the stream when autocloseBody is false")
+ public void shouldNotConsumeS3StreamObjectWhenIncludeBodyIsFalseAndNotCloseStreamWhenAutoCloseBodyIsFalse()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyFalseAutoCloseFalse");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyFalseAutoCloseFalse", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(ResponseInputStream.class)));
+ assertDoesNotThrow(() -> {
+ final ResponseInputStream<GetObjectResponse> inputStream = exchange.getIn().getBody(ResponseInputStream.class);
+ try (BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) {
+ final String text = reader.lines().collect(Collectors.joining());
+ assertThat(text, is("Test"));
+ }
+ });
+ }
+
+ @Test
+ @DisplayName("Should not consume S3StreamObject when include body is false and should close the stream when autocloseBody is true")
+ public void shouldNotConsumeS3StreamObjectWhenIncludeBodyIsFalseAndCloseStreamWhenAutoCloseBodyIsTrue()
+ throws InterruptedException {
+ result.reset();
+
+ result.expectedMessageCount(2);
+
+ template.setDefaultEndpointUri("direct:includeBodyFalseAutoCloseTrue");
+
+ template.send("direct:putObject", exchange -> {
+ exchange.getIn().setHeader(AWS2S3Constants.KEY, "test1.txt");
+ exchange.getIn().setBody("Test");
+ });
+
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(AWS2S3Constants.KEY, "test1.txt");
+ headers.put(Exchange.FILE_NAME, "test1.txt");
+
+ template.sendBodyAndHeaders("direct:includeBodyFalseAutoCloseTrue", headers);
+ result.assertIsSatisfied();
+
+ final Exchange exchange = result.getExchanges().get(1);
+
+ assertThat(exchange.getIn().getBody().getClass(), is(equalTo(ResponseInputStream.class)));
+ assertThrows(IOException.class, () -> {
+ final ResponseInputStream<GetObjectResponse> inputStream = exchange.getIn().getBody(ResponseInputStream.class);
+ inputStream.read();
+ });
+ }
+
@Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
@Override
public void configure() throws Exception {
+ String template = "aws2-s3://mycamel?autoCreateBucket=true&includeBody=%s&autocloseBody=%s";
+ String includeBodyTrueAutoCloseTrue = String.format(template, true, true);
+ String includeBodyFalseAutoCloseFalse = String.format(template, false, false);
+ String includeBodyFalseAutoCloseTrue = String.format(template, false, true);
+ from("direct:includeBodyTrueAutoCloseTrue").pollEnrich(includeBodyTrueAutoCloseTrue, 5000).to("mock:result");
+ from("direct:includeBodyFalseAutoCloseFalse").pollEnrich(includeBodyFalseAutoCloseFalse, 5000).to("mock:result");
+ from("direct:includeBodyFalseAutoCloseTrue").pollEnrich(includeBodyFalseAutoCloseTrue, 5000).to("mock:result");
+
String awsEndpoint = "aws2-s3://mycamel?autoCreateBucket=false";
from("direct:putObject").startupOrder(1).to(awsEndpoint).to("mock:result");
diff --git a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
index da54118..203b3b4 100644
--- a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
+++ b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
@@ -80,11 +80,11 @@ The AWS 2 S3 Storage Service component supports 42 options, which are listed bel
| *destinationBucketPrefix* (consumer) | Define the destination bucket prefix to use when an object must be moved and moveAfterRead is set to true. | | String
| *destinationBucketSuffix* (consumer) | Define the destination bucket suffix to use when an object must be moved and moveAfterRead is set to true. | | String
| *fileName* (consumer) | To get the object from the bucket with the given file name | | String
-| *includeBody* (consumer) | If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *includeBody* (consumer) | If it is true, the S3Object exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed in case of includeBody false then it will be up to the caller to close the S3Object stream [...]
| *includeFolders* (consumer) | If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those | true | boolean
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
-| *autocloseBody* (consumer) | If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
+| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]