[jira] [Created] (CAMEL-25789) camel-aws2-s3 - streaming upload with namingStrategy=random (or timestamp) names the first file of every producer keyName alone, so a restart replaces the first file of the previous run

["shashank (Jira)" <ji...@apache.org>]

shashank created CAMEL-25789:
--------------------------------

             Summary: camel-aws2-s3 - streaming upload with namingStrategy=random (or timestamp) names the first file of every producer keyName alone, so a restart replaces the first file of the previous run
                 Key: CAMEL-25789
                 URL: https://issues.apache.org/jira/browse/CAMEL-25789
             Project: Camel
          Issue Type: Bug
          Components: camel-aws2
            Reporter: shashank
            Assignee: shashank


{{AWS2S3StreamUploadProducer.fileNameToUpload}} (main 7aae27315946, lines 500-550) adds the UUID (namingStrategy {{random}}) or the timestamp ({{timestamp}}) to {{keyName}} only when {{part > 0}}:

{code:java}
case random:
    if (part > 0) {
        ... dynamicKeyName = fileName + "-" + id.toString() + ext;
    } else {
        ... dynamicKeyName = fileName + ext;
    }
{code}

{{part}} is the producer's counter of completed uploads ({{UploadState.part}}, line 256), which starts at 0 in every new producer. So the first file of every producer is named {{keyName}} alone ({{camel.txt}}), and only the later ones {{camel-<uuid>.txt}}. When the route or the application is started again, the new producer writes its first file to {{camel.txt}} again, and S3 replaces the first file of the previous run: its messages ({{batchMessageNumber}}, 10 by default, or up to {{batchSize}}) are lost at every restart, unless the bucket is versioned. The same happens between two producers with the same {{keyName}} (two instances of an application, or two routes). Every exchange completed normally.

The documentation (aws2-s3-streaming.adoc) says that "with the random strategy a UUID will be added after keyName", and that restarting from the last part "won't be needed when using the random naming strategy" (the {{restartingPolicy}} {{lastPart}} only works with {{progressive}}, whose {{override}} default replaces the earlier files by design). So a user who picks {{random}} to keep the files of every run loses one file per restart. {{AWSS3NamingStrategyTest}} pins the name of the first file ({{testFile.txt}} for part 0), but nothing explains why the first file should have no random part.

*Fix*: with {{random}} and {{timestamp}} every upload gets the UUID or the timestamp, the first one too. {{progressive}} is unchanged. Documentation of the three strategies in aws2-s3-streaming.adoc, and a note in the 4.24 upgrade guide (the first file has a new name).

*Tests*: new {{AWS2S3StreamUploadRandomNamingRestartTest}} (mocked {{S3Client}} that keeps the completed objects by key, like a bucket). {{aProducerStartedAgainDoesNotReplaceTheObjectsOfTheFirstOne}} (15 messages, stop, a new producer, 15 messages, stop) fails on main: {{every upload must be its own object, but the bucket has [camel-7c06...txt, camel-91f0...txt, camel.txt] ==> expected: <4> but was: <3>}}. {{everyObjectHasTheKeyNameAndARandomPart}} fails on main: {{camel.txt ==> expected: <true> but was: <false>}}. {{AWSS3NamingStrategyTest}} now expects the random part and the timestamp for part 0. camel-aws2-s3 unit tests: 106 run, 0 failures.

*How it was found*: variant analysis of CAMEL-25759 (pattern: a name that has to be unique across starts is built from per-start state, so a restart or a second instance reuses it), found while working on the timestamp grouping of the same producer. Then the JUnit test.

*Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x (the same {{part > 0}} condition; read, not run). The behaviour dates from CAMEL-16185. Not run against S3.

_Filed with Claude Code on behalf of allthingssecurity._




--
This message was sent by Atlassian Jira
(v8.20.10#820010)