You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2025/03/10 13:24:00 UTC

[jira] [Created] (HADOOP-19490) S3A: AWS SDK 2.30+ incompatible with third party stores

Steve Loughran created HADOOP-19490:
---------------------------------------

             Summary: S3A: AWS SDK 2.30+ incompatible with third party stores
                 Key: HADOOP-19490
                 URL: https://issues.apache.org/jira/browse/HADOOP-19490
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
            Reporter: Steve Loughran


Changes in the SDK related to a new AWS SDK Feature, Default integritY Protection {{https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html}} break all intereraction with third party S3 Stores.

See {{https://github.com/aws/aws-sdk-java-v2/issues/5801}}

There are documented mechanisms to turn this off
* an environment variable
* a change in ~/..aws/config
* a system property
* looks like a new builder option, though the docs don't cover it.

That checksum builder option looks like the only viable strategy, but it will need testing. maybe even make it a property which can be enabled/disabled, with tests and docs.

Whoever wants a 2.30.x feature either gets to do the fixing and testing. For 3.4.2 it'll be an older release.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org