You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2021/04/08 06:40:38 UTC
[camel] branch master updated (449b7a6 -> 807c199)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git.
from 449b7a6 TODO already fixed
new d80e245 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new 5e2c5a8 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new 29a803b CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new ab8eba6 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new 9353d45 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new 0001399 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
new 807c199 CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/camel/catalog/components/aws2-s3.json | 14 +++---
.../camel/catalog/docs/aws2-s3-component.adoc | 18 +++----
.../aws2/s3/AWS2S3ComponentConfigurer.java | 6 +++
.../aws2/s3/AWS2S3EndpointConfigurer.java | 6 +++
.../aws2/s3/AWS2S3EndpointUriFactory.java | 3 +-
.../apache/camel/component/aws2/s3/aws2-s3.json | 14 +++---
.../src/main/docs/aws2-s3-component.adoc | 18 +++----
.../component/aws2/s3/AWS2S3Configuration.java | 20 ++++++--
.../aws2/s3/stream/AWS2S3StreamUploadProducer.java | 33 +++++++++++++
...egyEnum.java => AWSS3RestartingPolicyEnum.java} | 6 +--
.../dsl/Aws2S3ComponentBuilderFactory.java | 24 ++++++++--
.../endpoint/dsl/AWS2S3EndpointBuilderFactory.java | 55 +++++++++++++++++++---
.../modules/ROOT/pages/aws2-s3-component.adoc | 18 +++----
13 files changed, 183 insertions(+), 52 deletions(-)
copy components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/{AWSS3NamingStrategyEnum.java => AWSS3RestartingPolicyEnum.java} (92%)
[camel] 01/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit d80e2450cff67187e83a82c3b4ba8ef841d3ac36
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 07:45:07 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../aws2/s3/stream/AWS2S3StreamUploadProducer.java | 38 +++++++++++++++-------
1 file changed, 27 insertions(+), 11 deletions(-)
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
index aa981eb..e83719b 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
@@ -19,6 +19,7 @@ package org.apache.camel.component.aws2.s3.stream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.util.ArrayList;
+import java.util.Comparator;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.ScheduledExecutorService;
@@ -38,17 +39,7 @@ import org.apache.camel.util.URISupport;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
-import software.amazon.awssdk.services.s3.model.AbortMultipartUploadRequest;
-import software.amazon.awssdk.services.s3.model.BucketCannedACL;
-import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest;
-import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadResponse;
-import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload;
-import software.amazon.awssdk.services.s3.model.CompletedPart;
-import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest;
-import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse;
-import software.amazon.awssdk.services.s3.model.ObjectCannedACL;
-import software.amazon.awssdk.services.s3.model.ServerSideEncryption;
-import software.amazon.awssdk.services.s3.model.UploadPartRequest;
+import software.amazon.awssdk.services.s3.model.*;
import software.amazon.awssdk.utils.IoUtils;
/**
@@ -85,6 +76,7 @@ public class AWS2S3StreamUploadProducer extends DefaultProducer {
getConfiguration().getStreamingUploadTimeout(), getConfiguration().getStreamingUploadTimeout(),
TimeUnit.MILLISECONDS);
}
+ setStartingPart();
}
@Override
@@ -278,6 +270,30 @@ public class AWS2S3StreamUploadProducer extends DefaultProducer {
return dynamicKeyName;
}
+ private void setStartingPart() {
+ if (getConfiguration().getNamingStrategy().equals(AWSS3NamingStrategyEnum.progressive)) {
+ ListObjectsRequest.Builder builder = ListObjectsRequest.builder().bucket(getConfiguration().getBucketName())
+ .prefix(AWS2S3Utils.determineFileName(getConfiguration().getKeyName()));
+ ListObjectsResponse o = getEndpoint().getS3Client().listObjects(builder.build());
+ if (o.contents().size() > 0) {
+ ArrayList<S3Object> list = new ArrayList<>();
+ list.addAll(o.contents());
+ list.sort(Comparator.comparing(S3Object::lastModified));
+ int listSize = list.size();
+ String fileName = AWS2S3Utils.determineFileName(list.get(listSize - 1).key());
+ int position = fileName.lastIndexOf("-");
+ if (position != -1) {
+ String partString = fileName.substring(position + 1);
+ if (ObjectHelper.isNotEmpty(partString)) {
+ part.getAndSet(Integer.valueOf(partString) + 1);
+ }
+ } else {
+ part.getAndSet(1);
+ }
+ }
+ }
+ }
+
protected AWS2S3Configuration getConfiguration() {
return getEndpoint().getConfiguration();
}
[camel] 02/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 5e2c5a8b6664015adb95800c3ab62d9c27a7f06a
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 07:51:39 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../camel/component/aws2/s3/AWS2S3Configuration.java | 20 +++++++++++++++++---
.../aws2/s3/stream/AWS2S3StreamUploadProducer.java | 6 +++++-
2 files changed, 22 insertions(+), 4 deletions(-)
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Configuration.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Configuration.java
index 0f48a66..2d2782e 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Configuration.java
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Configuration.java
@@ -18,6 +18,7 @@ package org.apache.camel.component.aws2.s3;
import org.apache.camel.RuntimeCamelException;
import org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum;
+import org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum;
import org.apache.camel.spi.Metadata;
import org.apache.camel.spi.UriParam;
import org.apache.camel.spi.UriParams;
@@ -121,6 +122,8 @@ public class AWS2S3Configuration implements Cloneable {
private AWSS3NamingStrategyEnum namingStrategy = AWSS3NamingStrategyEnum.progressive;
@UriParam(label = "producer")
private long streamingUploadTimeout;
+ @UriParam(defaultValue = "override", label = "producer")
+ private AWSS3RestartingPolicyEnum restartingPolicy = AWSS3RestartingPolicyEnum.override;
public long getPartSize() {
return partSize;
@@ -602,7 +605,7 @@ public class AWS2S3Configuration implements Cloneable {
}
/**
- * The number of messages composing a batch in stream mode
+ * The number of messages composing a batch in streaming upload mode
*/
public void setBatchMessageNumber(int batchMessageNumber) {
this.batchMessageNumber = batchMessageNumber;
@@ -613,7 +616,7 @@ public class AWS2S3Configuration implements Cloneable {
}
/**
- * The batch size (in bytes) in stream mode
+ * The batch size (in bytes) in streaming upload mode
*/
public void setBatchSize(int batchSize) {
this.batchSize = batchSize;
@@ -624,7 +627,7 @@ public class AWS2S3Configuration implements Cloneable {
}
/**
- * The naming strategy to use in stream mode
+ * The naming strategy to use in streaming upload mode
*/
public void setNamingStrategy(AWSS3NamingStrategyEnum namingStrategy) {
this.namingStrategy = namingStrategy;
@@ -641,6 +644,17 @@ public class AWS2S3Configuration implements Cloneable {
this.streamingUploadTimeout = streamingUploadTimeout;
}
+ public AWSS3RestartingPolicyEnum getRestartingPolicy() {
+ return restartingPolicy;
+ }
+
+ /**
+ * The restarting policy to use in streaming upload mode
+ */
+ public void setRestartingPolicy(AWSS3RestartingPolicyEnum restartingPolicy) {
+ this.restartingPolicy = restartingPolicy;
+ }
+
public AWS2S3Configuration copy() {
try {
return (AWS2S3Configuration) super.clone();
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
index e83719b..c9ce9d9 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
@@ -76,7 +76,9 @@ public class AWS2S3StreamUploadProducer extends DefaultProducer {
getConfiguration().getStreamingUploadTimeout(), getConfiguration().getStreamingUploadTimeout(),
TimeUnit.MILLISECONDS);
}
- setStartingPart();
+ if (getConfiguration().getRestartingPolicy().equals(AWSS3RestartingPolicyEnum.lastPart)) {
+ setStartingPart();
+ }
}
@Override
@@ -291,6 +293,8 @@ public class AWS2S3StreamUploadProducer extends DefaultProducer {
part.getAndSet(1);
}
}
+ } else {
+ LOG.info("lastPart restarting policy can be used only with progressive naming strategy");
}
}
[camel] 06/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 00013998d0b3fcd12513b36769d6202ede0014fa
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 08:27:01 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../aws2/s3/stream/AWS2S3StreamUploadProducer.java | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
index c9ce9d9..b089685 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
@@ -39,7 +39,20 @@ import org.apache.camel.util.URISupport;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
-import software.amazon.awssdk.services.s3.model.*;
+import software.amazon.awssdk.services.s3.model.AbortMultipartUploadRequest;
+import software.amazon.awssdk.services.s3.model.BucketCannedACL;
+import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadResponse;
+import software.amazon.awssdk.services.s3.model.CompletedPart;
+import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload;
+import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest;
+import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest;
+import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse;
+import software.amazon.awssdk.services.s3.model.ListObjectsRequest;
+import software.amazon.awssdk.services.s3.model.ListObjectsResponse;
+import software.amazon.awssdk.services.s3.model.ObjectCannedACL;
+import software.amazon.awssdk.services.s3.model.S3Object;
+import software.amazon.awssdk.services.s3.model.ServerSideEncryption;
+import software.amazon.awssdk.services.s3.model.UploadPartRequest;
import software.amazon.awssdk.utils.IoUtils;
/**
[camel] 03/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 29a803b596e2a9b4663f1c0a5070bdfdbb9dd5bc
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 07:51:54 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../aws2/s3/stream/AWSS3RestartingPolicyEnum.java | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWSS3RestartingPolicyEnum.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWSS3RestartingPolicyEnum.java
new file mode 100644
index 0000000..f2ccfae
--- /dev/null
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWSS3RestartingPolicyEnum.java
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.aws2.s3.stream;
+
+public enum AWSS3RestartingPolicyEnum {
+ override,
+ lastPart
+}
[camel] 07/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 807c1995a89d155b439ff1b76d6cbc3a6bfc69d0
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 08:39:46 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
index b089685..98e26a7 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
+++ b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/stream/AWS2S3StreamUploadProducer.java
@@ -41,10 +41,10 @@ import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.model.AbortMultipartUploadRequest;
import software.amazon.awssdk.services.s3.model.BucketCannedACL;
+import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest;
import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadResponse;
-import software.amazon.awssdk.services.s3.model.CompletedPart;
import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload;
-import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest;
+import software.amazon.awssdk.services.s3.model.CompletedPart;
import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest;
import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse;
import software.amazon.awssdk.services.s3.model.ListObjectsRequest;
[camel] 05/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 9353d452292625ada54e0a4bcf9c617cfe1b51ad
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 08:11:46 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../org/apache/camel/catalog/components/aws2-s3.json | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/aws2-s3.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/aws2-s3.json
index 584eec8..dd7871e 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/aws2-s3.json
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/aws2-s3.json
@@ -52,15 +52,16 @@
"moveAfterRead": { "kind": "property", "displayName": "Move After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish [...]
"prefix": { "kind": "property", "displayName": "Prefix", "group": "consumer", "label": "consumer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in." },
"autocloseBody": { "kind": "property", "displayName": "Autoclose Body", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If this option is true and includeBody is false, then the S3Object.close() met [...]
- "batchMessageNumber": { "kind": "property", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in stream mode" },
- "batchSize": { "kind": "property", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in stream mode" },
+ "batchMessageNumber": { "kind": "property", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in streaming upload mode" },
+ "batchSize": { "kind": "property", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in streaming upload mode" },
"deleteAfterWrite": { "kind": "property", "displayName": "Delete After Write", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Delete file object after the S3 file has been uploaded" },
"keyName": { "kind": "property", "displayName": "Key Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setting the key name for an element in the bucket through endpoint parameter" },
"lazyStartProducer": { "kind": "property", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during star [...]
"multiPartUpload": { "kind": "property", "displayName": "Multi Part Upload", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If it is true, camel will upload the file with multi part format, the part size is decided b [...]
- "namingStrategy": { "kind": "property", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descript [...]
+ "namingStrategy": { "kind": "property", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descript [...]
"operation": { "kind": "property", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.AWS2S3Operations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "getObjectRange" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": [...]
"partSize": { "kind": "property", "displayName": "Part Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 26214400, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setup the partSize which is used in multi part upload, the default size is 25M." },
+ "restartingPolicy": { "kind": "property", "displayName": "Restarting Policy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum", "enum": [ "override", "lastPart" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "override", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descri [...]
"storageClass": { "kind": "property", "displayName": "Storage Class", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request." },
"streamingUploadMode": { "kind": "property", "displayName": "Streaming Upload Mode", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "When stream mode is true the upload to bucket will be done in streaming" },
"streamingUploadTimeout": { "kind": "property", "displayName": "Streaming Upload Timeout", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "While streaming upload mode is true, this option set the timeout to complete upload" },
@@ -108,15 +109,16 @@
"exceptionHandler": { "kind": "parameter", "displayName": "Exception Handler", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.ExceptionHandler", "optionalPrefix": "consumer.", "deprecated": false, "autowired": false, "secret": false, "description": "To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the con [...]
"exchangePattern": { "kind": "parameter", "displayName": "Exchange Pattern", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.ExchangePattern", "enum": [ "InOnly", "InOut", "InOptionalOut" ], "deprecated": false, "autowired": false, "secret": false, "description": "Sets the exchange pattern when the consumer creates an exchange." },
"pollStrategy": { "kind": "parameter", "displayName": "Poll Strategy", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.PollingConsumerPollStrategy", "deprecated": false, "autowired": false, "secret": false, "description": "A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation [...]
- "batchMessageNumber": { "kind": "parameter", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in stream mode" },
- "batchSize": { "kind": "parameter", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in stream mode" },
+ "batchMessageNumber": { "kind": "parameter", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in streaming upload mode" },
+ "batchSize": { "kind": "parameter", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in streaming upload mode" },
"deleteAfterWrite": { "kind": "parameter", "displayName": "Delete After Write", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Delete file object after the S3 file has been uploaded" },
"keyName": { "kind": "parameter", "displayName": "Key Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setting the key name for an element in the bucket through endpoint parameter" },
"lazyStartProducer": { "kind": "parameter", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during sta [...]
"multiPartUpload": { "kind": "parameter", "displayName": "Multi Part Upload", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If it is true, camel will upload the file with multi part format, the part size is decided [...]
- "namingStrategy": { "kind": "parameter", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descrip [...]
+ "namingStrategy": { "kind": "parameter", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descrip [...]
"operation": { "kind": "parameter", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.AWS2S3Operations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "getObjectRange" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": [...]
"partSize": { "kind": "parameter", "displayName": "Part Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 26214400, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setup the partSize which is used in multi part upload, the default size is 25M." },
+ "restartingPolicy": { "kind": "parameter", "displayName": "Restarting Policy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum", "enum": [ "override", "lastPart" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "override", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descr [...]
"storageClass": { "kind": "parameter", "displayName": "Storage Class", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request." },
"streamingUploadMode": { "kind": "parameter", "displayName": "Streaming Upload Mode", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "When stream mode is true the upload to bucket will be done in streaming" },
"streamingUploadTimeout": { "kind": "parameter", "displayName": "Streaming Upload Timeout", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "While streaming upload mode is true, this option set the timeout to complete upload" },
[camel] 04/07: CAMEL-16469 - Camel-AWS2-S3 - Streaming upload:
restart from the last index when using the progressive naming strategy
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
commit ab8eba640aa5b6f5b2486451e71100acccdc9e5b
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Apr 8 08:03:06 2021 +0200
CAMEL-16469 - Camel-AWS2-S3 - Streaming upload: restart from the last index when using the progressive naming strategy
---
.../camel/catalog/docs/aws2-s3-component.adoc | 18 +++----
.../aws2/s3/AWS2S3ComponentConfigurer.java | 6 +++
.../aws2/s3/AWS2S3EndpointConfigurer.java | 6 +++
.../aws2/s3/AWS2S3EndpointUriFactory.java | 3 +-
.../apache/camel/component/aws2/s3/aws2-s3.json | 14 +++---
.../src/main/docs/aws2-s3-component.adoc | 18 +++----
.../dsl/Aws2S3ComponentBuilderFactory.java | 24 ++++++++--
.../endpoint/dsl/AWS2S3EndpointBuilderFactory.java | 55 +++++++++++++++++++---
.../modules/ROOT/pages/aws2-s3-component.adoc | 18 +++----
9 files changed, 122 insertions(+), 40 deletions(-)
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
index b4dd75b..188dab6 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/aws2-s3-component.adoc
@@ -47,7 +47,7 @@ from("aws2-s3://helloBucket?accessKey=yourAccessKey&secretKey=yourSecretKey&pref
// component options: START
-The AWS 2 S3 Storage Service component supports 48 options, which are listed below.
+The AWS 2 S3 Storage Service component supports 49 options, which are listed below.
@@ -84,15 +84,16 @@ The AWS 2 S3 Storage Service component supports 48 options, which are listed bel
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long
@@ -134,7 +135,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (66 parameters):
+=== Query Parameters (67 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -175,15 +176,16 @@ with the following path and query parameters:
| *exceptionHandler* (consumer) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
| *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern
| *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long
diff --git a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3ComponentConfigurer.java b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3ComponentConfigurer.java
index 4920a0a..9ff15ea 100644
--- a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3ComponentConfigurer.java
+++ b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3ComponentConfigurer.java
@@ -100,6 +100,8 @@ public class AWS2S3ComponentConfigurer extends PropertyConfigurerSupport impleme
case "proxyprotocol":
case "proxyProtocol": getOrCreateConfiguration(target).setProxyProtocol(property(camelContext, software.amazon.awssdk.core.Protocol.class, value)); return true;
case "region": getOrCreateConfiguration(target).setRegion(property(camelContext, java.lang.String.class, value)); return true;
+ case "restartingpolicy":
+ case "restartingPolicy": getOrCreateConfiguration(target).setRestartingPolicy(property(camelContext, org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum.class, value)); return true;
case "secretkey":
case "secretKey": getOrCreateConfiguration(target).setSecretKey(property(camelContext, java.lang.String.class, value)); return true;
case "storageclass":
@@ -202,6 +204,8 @@ public class AWS2S3ComponentConfigurer extends PropertyConfigurerSupport impleme
case "proxyprotocol":
case "proxyProtocol": return software.amazon.awssdk.core.Protocol.class;
case "region": return java.lang.String.class;
+ case "restartingpolicy":
+ case "restartingPolicy": return org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum.class;
case "secretkey":
case "secretKey": return java.lang.String.class;
case "storageclass":
@@ -300,6 +304,8 @@ public class AWS2S3ComponentConfigurer extends PropertyConfigurerSupport impleme
case "proxyprotocol":
case "proxyProtocol": return getOrCreateConfiguration(target).getProxyProtocol();
case "region": return getOrCreateConfiguration(target).getRegion();
+ case "restartingpolicy":
+ case "restartingPolicy": return getOrCreateConfiguration(target).getRestartingPolicy();
case "secretkey":
case "secretKey": return getOrCreateConfiguration(target).getSecretKey();
case "storageclass":
diff --git a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointConfigurer.java b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointConfigurer.java
index 9b57190..8f689c7 100644
--- a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointConfigurer.java
+++ b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointConfigurer.java
@@ -112,6 +112,8 @@ public class AWS2S3EndpointConfigurer extends PropertyConfigurerSupport implemen
case "region": target.getConfiguration().setRegion(property(camelContext, java.lang.String.class, value)); return true;
case "repeatcount":
case "repeatCount": target.setRepeatCount(property(camelContext, long.class, value)); return true;
+ case "restartingpolicy":
+ case "restartingPolicy": target.getConfiguration().setRestartingPolicy(property(camelContext, org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum.class, value)); return true;
case "runlogginglevel":
case "runLoggingLevel": target.setRunLoggingLevel(property(camelContext, org.apache.camel.LoggingLevel.class, value)); return true;
case "scheduledexecutorservice":
@@ -248,6 +250,8 @@ public class AWS2S3EndpointConfigurer extends PropertyConfigurerSupport implemen
case "region": return java.lang.String.class;
case "repeatcount":
case "repeatCount": return long.class;
+ case "restartingpolicy":
+ case "restartingPolicy": return org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum.class;
case "runlogginglevel":
case "runLoggingLevel": return org.apache.camel.LoggingLevel.class;
case "scheduledexecutorservice":
@@ -380,6 +384,8 @@ public class AWS2S3EndpointConfigurer extends PropertyConfigurerSupport implemen
case "region": return target.getConfiguration().getRegion();
case "repeatcount":
case "repeatCount": return target.getRepeatCount();
+ case "restartingpolicy":
+ case "restartingPolicy": return target.getConfiguration().getRestartingPolicy();
case "runlogginglevel":
case "runLoggingLevel": return target.getRunLoggingLevel();
case "scheduledexecutorservice":
diff --git a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointUriFactory.java b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointUriFactory.java
index 6eb9073..6debae3 100644
--- a/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointUriFactory.java
+++ b/components/camel-aws/camel-aws2-s3/src/generated/java/org/apache/camel/component/aws2/s3/AWS2S3EndpointUriFactory.java
@@ -20,7 +20,7 @@ public class AWS2S3EndpointUriFactory extends org.apache.camel.support.component
private static final Set<String> PROPERTY_NAMES;
private static final Set<String> SECRET_PROPERTY_NAMES;
static {
- Set<String> props = new HashSet<>(67);
+ Set<String> props = new HashSet<>(68);
props.add("customerAlgorithm");
props.add("fileName");
props.add("batchMessageNumber");
@@ -57,6 +57,7 @@ public class AWS2S3EndpointUriFactory extends org.apache.camel.support.component
props.add("deleteAfterWrite");
props.add("region");
props.add("exceptionHandler");
+ props.add("restartingPolicy");
props.add("backoffMultiplier");
props.add("destinationBucket");
props.add("amazonS3Presigner");
diff --git a/components/camel-aws/camel-aws2-s3/src/generated/resources/org/apache/camel/component/aws2/s3/aws2-s3.json b/components/camel-aws/camel-aws2-s3/src/generated/resources/org/apache/camel/component/aws2/s3/aws2-s3.json
index 584eec8..dd7871e 100644
--- a/components/camel-aws/camel-aws2-s3/src/generated/resources/org/apache/camel/component/aws2/s3/aws2-s3.json
+++ b/components/camel-aws/camel-aws2-s3/src/generated/resources/org/apache/camel/component/aws2/s3/aws2-s3.json
@@ -52,15 +52,16 @@
"moveAfterRead": { "kind": "property", "displayName": "Move After Read", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish [...]
"prefix": { "kind": "property", "displayName": "Prefix", "group": "consumer", "label": "consumer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in." },
"autocloseBody": { "kind": "property", "displayName": "Autoclose Body", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If this option is true and includeBody is false, then the S3Object.close() met [...]
- "batchMessageNumber": { "kind": "property", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in stream mode" },
- "batchSize": { "kind": "property", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in stream mode" },
+ "batchMessageNumber": { "kind": "property", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in streaming upload mode" },
+ "batchSize": { "kind": "property", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in streaming upload mode" },
"deleteAfterWrite": { "kind": "property", "displayName": "Delete After Write", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Delete file object after the S3 file has been uploaded" },
"keyName": { "kind": "property", "displayName": "Key Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setting the key name for an element in the bucket through endpoint parameter" },
"lazyStartProducer": { "kind": "property", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during star [...]
"multiPartUpload": { "kind": "property", "displayName": "Multi Part Upload", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If it is true, camel will upload the file with multi part format, the part size is decided b [...]
- "namingStrategy": { "kind": "property", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descript [...]
+ "namingStrategy": { "kind": "property", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descript [...]
"operation": { "kind": "property", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.AWS2S3Operations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "getObjectRange" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": [...]
"partSize": { "kind": "property", "displayName": "Part Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 26214400, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setup the partSize which is used in multi part upload, the default size is 25M." },
+ "restartingPolicy": { "kind": "property", "displayName": "Restarting Policy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum", "enum": [ "override", "lastPart" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "override", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descri [...]
"storageClass": { "kind": "property", "displayName": "Storage Class", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request." },
"streamingUploadMode": { "kind": "property", "displayName": "Streaming Upload Mode", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "When stream mode is true the upload to bucket will be done in streaming" },
"streamingUploadTimeout": { "kind": "property", "displayName": "Streaming Upload Timeout", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "While streaming upload mode is true, this option set the timeout to complete upload" },
@@ -108,15 +109,16 @@
"exceptionHandler": { "kind": "parameter", "displayName": "Exception Handler", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.ExceptionHandler", "optionalPrefix": "consumer.", "deprecated": false, "autowired": false, "secret": false, "description": "To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the con [...]
"exchangePattern": { "kind": "parameter", "displayName": "Exchange Pattern", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.ExchangePattern", "enum": [ "InOnly", "InOut", "InOptionalOut" ], "deprecated": false, "autowired": false, "secret": false, "description": "Sets the exchange pattern when the consumer creates an exchange." },
"pollStrategy": { "kind": "parameter", "displayName": "Poll Strategy", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.PollingConsumerPollStrategy", "deprecated": false, "autowired": false, "secret": false, "description": "A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation [...]
- "batchMessageNumber": { "kind": "parameter", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in stream mode" },
- "batchSize": { "kind": "parameter", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in stream mode" },
+ "batchMessageNumber": { "kind": "parameter", "displayName": "Batch Message Number", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The number of messages composing a batch in streaming upload mode" },
+ "batchSize": { "kind": "parameter", "displayName": "Batch Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000000, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The batch size (in bytes) in streaming upload mode" },
"deleteAfterWrite": { "kind": "parameter", "displayName": "Delete After Write", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Delete file object after the S3 file has been uploaded" },
"keyName": { "kind": "parameter", "displayName": "Key Name", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setting the key name for an element in the bucket through endpoint parameter" },
"lazyStartProducer": { "kind": "parameter", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during sta [...]
"multiPartUpload": { "kind": "parameter", "displayName": "Multi Part Upload", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "If it is true, camel will upload the file with multi part format, the part size is decided [...]
- "namingStrategy": { "kind": "parameter", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descrip [...]
+ "namingStrategy": { "kind": "parameter", "displayName": "Naming Strategy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum", "enum": [ "progressive", "random" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "progressive", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descrip [...]
"operation": { "kind": "parameter", "displayName": "Operation", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.AWS2S3Operations", "enum": [ "copyObject", "listObjects", "deleteObject", "deleteBucket", "listBuckets", "getObject", "getObjectRange" ], "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": [...]
"partSize": { "kind": "parameter", "displayName": "Part Size", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 26214400, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "Setup the partSize which is used in multi part upload, the default size is 25M." },
+ "restartingPolicy": { "kind": "parameter", "displayName": "Restarting Policy", "group": "producer", "label": "producer", "required": false, "type": "object", "javaType": "org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum", "enum": [ "override", "lastPart" ], "deprecated": false, "autowired": false, "secret": false, "defaultValue": "override", "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "descr [...]
"storageClass": { "kind": "parameter", "displayName": "Storage Class", "group": "producer", "label": "producer", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request." },
"streamingUploadMode": { "kind": "parameter", "displayName": "Streaming Upload Mode", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "When stream mode is true the upload to bucket will be done in streaming" },
"streamingUploadTimeout": { "kind": "parameter", "displayName": "Streaming Upload Timeout", "group": "producer", "label": "producer", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "configurationClass": "org.apache.camel.component.aws2.s3.AWS2S3Configuration", "configurationField": "configuration", "description": "While streaming upload mode is true, this option set the timeout to complete upload" },
diff --git a/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc b/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
index b4dd75b..188dab6 100644
--- a/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
+++ b/components/camel-aws/camel-aws2-s3/src/main/docs/aws2-s3-component.adoc
@@ -47,7 +47,7 @@ from("aws2-s3://helloBucket?accessKey=yourAccessKey&secretKey=yourSecretKey&pref
// component options: START
-The AWS 2 S3 Storage Service component supports 48 options, which are listed below.
+The AWS 2 S3 Storage Service component supports 49 options, which are listed below.
@@ -84,15 +84,16 @@ The AWS 2 S3 Storage Service component supports 48 options, which are listed bel
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long
@@ -134,7 +135,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (66 parameters):
+=== Query Parameters (67 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -175,15 +176,16 @@ with the following path and query parameters:
| *exceptionHandler* (consumer) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
| *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern
| *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long
diff --git a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/Aws2S3ComponentBuilderFactory.java b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/Aws2S3ComponentBuilderFactory.java
index 9218000..8af4047 100644
--- a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/Aws2S3ComponentBuilderFactory.java
+++ b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/Aws2S3ComponentBuilderFactory.java
@@ -557,7 +557,7 @@ public interface Aws2S3ComponentBuilderFactory {
return this;
}
/**
- * The number of messages composing a batch in stream mode.
+ * The number of messages composing a batch in streaming upload mode.
*
* The option is a: <code>int</code> type.
*
@@ -572,7 +572,7 @@ public interface Aws2S3ComponentBuilderFactory {
return this;
}
/**
- * The batch size (in bytes) in stream mode.
+ * The batch size (in bytes) in streaming upload mode.
*
* The option is a: <code>int</code> type.
*
@@ -657,7 +657,7 @@ public interface Aws2S3ComponentBuilderFactory {
return this;
}
/**
- * The naming strategy to use in stream mode.
+ * The naming strategy to use in streaming upload mode.
*
* The option is a:
* <code>org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum</code> type.
@@ -706,6 +706,23 @@ public interface Aws2S3ComponentBuilderFactory {
return this;
}
/**
+ * The restarting policy to use in streaming upload mode.
+ *
+ * The option is a:
+ * <code>org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum</code> type.
+ *
+ * Default: override
+ * Group: producer
+ *
+ * @param restartingPolicy the value to set
+ * @return the dsl builder
+ */
+ default Aws2S3ComponentBuilder restartingPolicy(
+ org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum restartingPolicy) {
+ doSetProperty("restartingPolicy", restartingPolicy);
+ return this;
+ }
+ /**
* The storage class to set in the
* com.amazonaws.services.s3.model.PutObjectRequest request.
*
@@ -909,6 +926,7 @@ public interface Aws2S3ComponentBuilderFactory {
case "namingStrategy": getOrCreateConfiguration((AWS2S3Component) component).setNamingStrategy((org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum) value); return true;
case "operation": getOrCreateConfiguration((AWS2S3Component) component).setOperation((org.apache.camel.component.aws2.s3.AWS2S3Operations) value); return true;
case "partSize": getOrCreateConfiguration((AWS2S3Component) component).setPartSize((long) value); return true;
+ case "restartingPolicy": getOrCreateConfiguration((AWS2S3Component) component).setRestartingPolicy((org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum) value); return true;
case "storageClass": getOrCreateConfiguration((AWS2S3Component) component).setStorageClass((java.lang.String) value); return true;
case "streamingUploadMode": getOrCreateConfiguration((AWS2S3Component) component).setStreamingUploadMode((boolean) value); return true;
case "streamingUploadTimeout": getOrCreateConfiguration((AWS2S3Component) component).setStreamingUploadTimeout((long) value); return true;
diff --git a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/AWS2S3EndpointBuilderFactory.java b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/AWS2S3EndpointBuilderFactory.java
index bcc7ff6..89e3638 100644
--- a/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/AWS2S3EndpointBuilderFactory.java
+++ b/core/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/AWS2S3EndpointBuilderFactory.java
@@ -1937,7 +1937,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The number of messages composing a batch in stream mode.
+ * The number of messages composing a batch in streaming upload mode.
*
* The option is a: <code>int</code> type.
*
@@ -1953,7 +1953,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The number of messages composing a batch in stream mode.
+ * The number of messages composing a batch in streaming upload mode.
*
* The option will be converted to a <code>int</code> type.
*
@@ -1969,7 +1969,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The batch size (in bytes) in stream mode.
+ * The batch size (in bytes) in streaming upload mode.
*
* The option is a: <code>int</code> type.
*
@@ -1984,7 +1984,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The batch size (in bytes) in stream mode.
+ * The batch size (in bytes) in streaming upload mode.
*
* The option will be converted to a <code>int</code> type.
*
@@ -2131,7 +2131,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The naming strategy to use in stream mode.
+ * The naming strategy to use in streaming upload mode.
*
* The option is a:
* <code>org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum</code> type.
@@ -2148,7 +2148,7 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
- * The naming strategy to use in stream mode.
+ * The naming strategy to use in streaming upload mode.
*
* The option will be converted to a
* <code>org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum</code> type.
@@ -2228,6 +2228,40 @@ public interface AWS2S3EndpointBuilderFactory {
return this;
}
/**
+ * The restarting policy to use in streaming upload mode.
+ *
+ * The option is a:
+ * <code>org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum</code> type.
+ *
+ * Default: override
+ * Group: producer
+ *
+ * @param restartingPolicy the value to set
+ * @return the dsl builder
+ */
+ default AWS2S3EndpointProducerBuilder restartingPolicy(
+ AWSS3RestartingPolicyEnum restartingPolicy) {
+ doSetProperty("restartingPolicy", restartingPolicy);
+ return this;
+ }
+ /**
+ * The restarting policy to use in streaming upload mode.
+ *
+ * The option will be converted to a
+ * <code>org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum</code> type.
+ *
+ * Default: override
+ * Group: producer
+ *
+ * @param restartingPolicy the value to set
+ * @return the dsl builder
+ */
+ default AWS2S3EndpointProducerBuilder restartingPolicy(
+ String restartingPolicy) {
+ doSetProperty("restartingPolicy", restartingPolicy);
+ return this;
+ }
+ /**
* The storage class to set in the
* com.amazonaws.services.s3.model.PutObjectRequest request.
*
@@ -2962,6 +2996,15 @@ public interface AWS2S3EndpointBuilderFactory {
createDownloadLink;
}
+ /**
+ * Proxy enum for
+ * <code>org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum</code> enum.
+ */
+ enum AWSS3RestartingPolicyEnum {
+ override,
+ lastPart;
+ }
+
public interface AWS2S3Builders {
/**
* AWS 2 S3 Storage Service (camel-aws2-s3)
diff --git a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
index a895c3a..0bfe40c 100644
--- a/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
+++ b/docs/components/modules/ROOT/pages/aws2-s3-component.adoc
@@ -49,7 +49,7 @@ from("aws2-s3://helloBucket?accessKey=yourAccessKey&secretKey=yourSecretKey&pref
// component options: START
-The AWS 2 S3 Storage Service component supports 48 options, which are listed below.
+The AWS 2 S3 Storage Service component supports 49 options, which are listed below.
@@ -86,15 +86,16 @@ The AWS 2 S3 Storage Service component supports 48 options, which are listed bel
| *moveAfterRead* (consumer) | Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved. | false | boolean
| *prefix* (consumer) | The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in. | | String
| *autocloseBody* (consumer) | If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically. | true | boolean
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long
@@ -136,7 +137,7 @@ with the following path and query parameters:
|===
-=== Query Parameters (66 parameters):
+=== Query Parameters (67 parameters):
[width="100%",cols="2,5,^1,2",options="header"]
@@ -177,15 +178,16 @@ with the following path and query parameters:
| *exceptionHandler* (consumer) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. | | ExceptionHandler
| *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut | | ExchangePattern
| *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | | PollingConsumerPollStrategy
-| *batchMessageNumber* (producer) | The number of messages composing a batch in stream mode | 10 | int
-| *batchSize* (producer) | The batch size (in bytes) in stream mode | 1000000 | int
+| *batchMessageNumber* (producer) | The number of messages composing a batch in streaming upload mode | 10 | int
+| *batchSize* (producer) | The batch size (in bytes) in streaming upload mode | 1000000 | int
| *deleteAfterWrite* (producer) | Delete file object after the S3 file has been uploaded | false | boolean
| *keyName* (producer) | Setting the key name for an element in the bucket through endpoint parameter | | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and [...]
| *multiPartUpload* (producer) | If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize | false | boolean
-| *namingStrategy* (producer) | The naming strategy to use in stream mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
+| *namingStrategy* (producer) | The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of: progressive, random | progressive | AWSS3NamingStrategyEnum
| *operation* (producer) | The operation to do in case the user don't want to do only an upload. There are 7 enums and the value can be one of: copyObject, listObjects, deleteObject, deleteBucket, listBuckets, getObject, getObjectRange | | AWS2S3Operations
| *partSize* (producer) | Setup the partSize which is used in multi part upload, the default size is 25M. | 26214400 | long
+| *restartingPolicy* (producer) | The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of: override, lastPart | override | AWSS3RestartingPolicyEnum
| *storageClass* (producer) | The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request. | | String
| *streamingUploadMode* (producer) | When stream mode is true the upload to bucket will be done in streaming | false | boolean
| *streamingUploadTimeout* (producer) | While streaming upload mode is true, this option set the timeout to complete upload | | long