[GitHub] [camel-quarkus] jingwang opened a new issue, #3992: aws S3 streaming upload random naming strategy always overwrite the initial file (without suffix)

[GitBox <gi...@apache.org>]

jingwang opened a new issue, #3992:
URL: https://github.com/apache/camel-quarkus/issues/3992

   When choosing `namingStrategy(AWSS3NamingStrategyEnum.random)` and `restartingPolicy(AWSS3RestartingPolicyEnum.lastPart)`, the very first file without any UUID suffix is always overwritten when the application restarts. One would expect in this case it would start with a file with a new UUID suffix.
   
   version:
   ```
           <camel-quarkus.version>2.11.0</camel-quarkus.version>
           <quarkus-plugin.version>2.11.1.Final</quarkus-plugin.version>
           <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
           <quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
           <quarkus.platform.version>2.11.1.Final</quarkus.platform.version>
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-quarkus] jingwang commented on issue #3992: aws S3 streaming upload random naming strategy always overwrite the initial file (without suffix)

[GitBox <gi...@apache.org>]

jingwang commented on issue #3992:
URL: https://github.com/apache/camel-quarkus/issues/3992#issuecomment-1214556743

   Thanks @zhfeng . Here is the test with docker-compose using localstack:
   
   **docker-compose.yaml**
   ```
   version: '3.8'
   services:
     localstack:
       image: localstack/localstack:latest
       environment:
         - AWS_DEFAULT_REGION=us-west-2
         - EDGE_PORT=4566
         - SERVICES=s3
       ports:
         - '4566-4583:4566-4583'
   ```
   **Aws2S3Route.java**
   ```
   import org.apache.camel.builder.endpoint.EndpointRouteBuilder;
   import org.apache.camel.builder.endpoint.dsl.AWS2S3EndpointBuilderFactory;
   import org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum;
   import org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum;
   import javax.enterprise.context.ApplicationScoped;
   
   @ApplicationScoped
   public class Aws2S3Route extends EndpointRouteBuilder {
       @Override
       public void configure() throws Exception {
   
               AWS2S3EndpointBuilderFactory.AWS2S3EndpointProducerBuilder builder = aws2S3("bucket1")
                       .region("us-west-2")
                       .autoCreateBucket(true)
                       .useDefaultCredentialsProvider(true)
                       .streamingUploadMode(true)
                       .overrideEndpoint(true)
                       .uriEndpointOverride("http://localhost:4566")
                       .batchMessageNumber(2) // 20 messages
                       .streamingUploadTimeout(5000) // or after 10 minutes
                       .namingStrategy(AWSS3NamingStrategyEnum.random)
                       .restartingPolicy(AWSS3RestartingPolicyEnum.lastPart)
                       .keyName("file.txt");
   
               from(cron("test").schedule("0/5 * * * * ?"))
                       .setBody(constant("test message"))
                       .to(builder);
       }
   }
   ```
   **pom.xml**
   ```
   ...
           <camel-quarkus.version>2.11.0</camel-quarkus.version>
           <quarkus-plugin.version>2.11.1.Final</quarkus-plugin.version>
           <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
           <quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
           <quarkus.platform.version>2.11.1.Final</quarkus.platform.version>
   ...
           <dependency>
               <groupId>org.apache.camel.quarkus</groupId>
               <artifactId>camel-quarkus-aws2-s3</artifactId>
           </dependency>
   
           <dependency>
               <groupId>org.apache.camel.quarkus</groupId>
               <artifactId>camel-quarkus-cron</artifactId>
           </dependency>
   ```
   
   **Steps to reproduce:**
   
   - Start docker-compose
   ```
   > docker-compose up
   ```
   - On a separate tab, start application
   ```
   > export AWS_ACCESS_KEY_ID=accesskey
   > export AWS_SECRET_ACCESS_KEY=secretkey
   > mvn clean quarkus:dev
   ```
   - On a separate tab, monitor the s3 bucket and check the timestamp of `file.txt`
   ```
   aws --endpoint-url=http://localhost:4566 s3 ls s3://bucket1 --recursive
   ```
   **Expected:**
   When the application is restarted, the timestamp of `file.txt` should not change (i.e., it is not overwritten)
   **Actual:**
   Every time when the application is restarted, the timestamp of `file.txt` changes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-quarkus] zhfeng commented on issue #3992: aws S3 streaming upload random naming strategy always overwrite the initial file (without suffix)

[GitBox <gi...@apache.org>]

zhfeng commented on issue #3992:
URL: https://github.com/apache/camel-quarkus/issues/3992#issuecomment-1214525916

   @jingwang Thanks for reporting! - can you provide a reproducer or unit test for this issue? I will take a look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org