You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by "quantranhong1999 (via GitHub)" <gi...@apache.org> on 2025/06/10 09:45:09 UTC

[PR] JAMES-4137 Upgrade S3 SDK to 2.30.x onwards [james-project]

quantranhong1999 opened a new pull request, #2739:
URL: https://github.com/apache/james-project/pull/2739

   In S3 SDK version 2.30.0, AWS team introduced a breaking change in the integrity check in the driver:
   
   - Checksum is always required by default by AWS SDK >= 2.30.x
   - AWS SDK >= 2.30.x does not send MD5 checksum anymore (which 3rd party S3 storages rely on), but relies on CRC32 checksum.
   
   cf: https://github.com/aws/aws-sdk-java-v2/discussions/5802
   
   This breaks the compatibility with 3rd party object storage(s) that still rely on MD5 checksum. We failed to upgrade the SDK in the past: https://github.com/apache/james-project/pull/2638.
   
   It seems after receiving a lot of complaints from the community (Apache Hadoop, Apache Iceberg, Apache Spark...), AWS has agreed to have an option to maintain backward compatibility with S3-compatible storages. cf https://github.com/aws/aws-sdk-java-v2/pull/6055.
   
   Apache Iceberg is starting to adopt the backward compatibility option: https://github.com/apache/iceberg/pull/12264.
   
   Let's try to adopt this on our side, with care.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org

Re: [PR] JAMES-4137 Upgrade S3 SDK to 2.30.x onwards [james-project]

Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2739:
URL: https://github.com/apache/james-project/pull/2739#issuecomment-2961010265

   > the backward compatibility wasn't added yet when i had a look at this before> Good catch!!!
   
   Yes, it has been introduced recently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org

Re: [PR] JAMES-4137 Upgrade S3 SDK to 2.30.x onwards [james-project]

Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2739:
URL: https://github.com/apache/james-project/pull/2739#issuecomment-2958425650

   Tests with Scality and MinIO passed (failed without this backward option).
   
   I would love to double check this against OVH S3.
   
   Maybe we can put this PR on hold for a while, so the community can review and try against their S3 storage solution, if needed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org

Re: [PR] JAMES-4137 Upgrade S3 SDK to 2.30.x onwards [james-project]

Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa merged PR #2739:
URL: https://github.com/apache/james-project/pull/2739


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org