[jira] [Created] (CAMEL-24156) camel-aws2 - Umbrella: high-severity findings from July 2026 review (S3, SQS, SNS, Kinesis, DDB)

["Federico Mariani (Jira)" <ji...@apache.org>]

Federico Mariani created CAMEL-24156:
----------------------------------------

             Summary: camel-aws2 - Umbrella: high-severity findings from July 2026 review (S3, SQS, SNS, Kinesis, DDB)
                 Key: CAMEL-24156
                 URL: https://issues.apache.org/jira/browse/CAMEL-24156
             Project: Camel
          Issue Type: Bug
          Components: camel-aws
            Reporter: Federico Mariani
         Attachments: camel-aws-review-2026-07.md

Umbrella for the high-severity findings of a deep code review of camel-aws2-s3, -sqs, -sns, -kinesis (incl. firehose) and -ddb (incl. ddbstream) on main (4.22.0-SNAPSHOT). The full findings document (with failure scenarios and suggested fixes per item) is attached. The two critical findings are tracked separately with failing reproducer tests attached: the S3 streaming truncation and the DDB Streams resharding data loss.

Individual findings can be split out into their own issues as they are picked up.

*camel-aws2-s3*
# {{AWS2S3Producer:645}} — {{operation=getObject&ignoreBody=true}} leaks the ResponseInputStream; connection pool exhausted after ~50 exchanges (CAMEL-21116 follow-up).
# {{AWS2S3Producer:222-305}} — {{processMultiPart}} computes the dynamic bucket name ({{CamelAwsS3OverrideBucketName}} / simple expressions) but builds all four multipart requests with the raw configured bucket; behavior silently diverges by payload size.
# {{AWS2S3Producer:187-217}} — FileInputStream opened for File bodies is never closed on the {{contentLength < partSize}} shortcut; one fd leaked per small file in multiPartUpload mode.

*camel-aws2-sqs*
# {{Sqs2Producer:270-300}} — per-message {{DelaySeconds}} is applied even when {{delayQueue=true}}; on FIFO queues every send fails with InvalidParameterValue, on standard queues the queue-level option is defeated.

*camel-aws2-sns*
# {{Sns2Endpoint:147-150}} — auto-created FIFO topics never get {{ContentBasedDeduplication}}, so {{messageDeduplicationIdStrategy=useContentBasedDeduplication}} makes every publish fail (SQS twin handles this correctly).

*camel-aws2-kinesis*
# {{Kinesis2Configuration:71}} — {{shardClosed}} default is only declared in the annotation; the field is null and {{handleClosedShard}} NPEs on the first shard rotation with default config.
# {{Kinesis2Consumer:176-297}} — expired shard iterators are never invalidated (no ExpiredIteratorException handling in the module); any >5-minute gap stalls the consumer permanently.
# {{KinesisResumeAction:62-67}} + {{KinesisResumeStrategy:60-63}} — resume support broken for multi-shard streams: evalEntry overwrites the request with every cache entry, KinesisOffset values hit a String cast (ClassCastException), and the single registry action is mutated concurrently from parallelStream.
# {{KinesisConnection:38-64}} + {{Kinesis2Endpoint:80-93}} — the component-wide connection caches the first endpoint's client for all endpoints (second endpoint's region/credentials silently ignored) and {{doStop}} closes the shared client without resetting the cache.
# {{KclKinesis2Consumer:193-232}} — KCL consumer never checkpoints during processing, swallows all processing Throwables (not even logged with stack trace), then checkpoints unconditionally at shutdown — data loss on failure, mass reprocessing on crash.
# {{KclKinesis2Consumer:94-133}} — session-credentials branch is unreachable ({{accessKey && secretKey && sessionToken}} tested after {{accessKey && secretKey}}); KCL DynamoDB/CloudWatch clients fail with STS temporary credentials.

*camel-aws2-ddb (ddbstream)*
# {{Ddb2StreamConsumer:65,93}} + {{ShardIteratorHandler:77-86}} — internal iterator map returned by reference and structurally modified during poll iteration -> ConcurrentModificationException whenever a shard retires while >=2 shards are tracked.
# {{ShardIteratorHandler:55-57}} — DescribeStream pagination ({{lastEvaluatedShardId}}) ignored; streams with >100 shards silently lose partitions.
# {{ShardIteratorHandler:88-92}} — expired-iterator recovery sends {{AFTER_SEQUENCE_NUMBER}} with null sequence when no record was seen; consumer permanently stuck failing.

*camel-aws2-ddb (transformer)*
# {{Ddb2JsonDataTypeTransformer:197-247}} — Long/BigInteger/BigDecimal values (e.g. epoch millis) and numeric arrays are stored as DynamoDB String/StringSet types instead of N/NS (CAMEL-22038 only fixed scalar Double).

_This issue was researched and filed by Claude Code (AI) on behalf of Federico Mariani (fmariani). Full findings document from the review is attached to the umbrella issues._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)