You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ozone.apache.org by Wei-Chiu Chuang <we...@apache.org> on 2024/10/03 20:50:08 UTC
Do we test against Hadoop S3A?
Hi I know Ozone's acceptance test verifies aws s3 cli against Ozone S3
Gateway.
Do we test against Hadoop S3A client? Hadoop community is reporting a
regression due to AWS v2 SDK (*) so I'm wondering if we have the test
coverage to ensure compatibility.
In addition, do we test or do we plan to test AWS cli v2
* Hadoop 3.4 updated to AWS SDK v2.
Re: Do we test against Hadoop S3A?
Posted by Ivan Andika <iv...@rocketmail.com.INVALID>.
Hi Wei Chiu,
Regarding testing Ozone support AWS SDK, I’m recently continued my work on supporting S3G in Ozone integration test environment (https://issues.apache.org/jira/browse/HDDS-10390) to support AWS S3 SDK tests. I’m currently writing some tests using AWS V1 SDK since it’s the one used by Ozone, but we can import and add AWS V2 SDK tests in the future. I expect that we can also add the Hadoop S3A client support on the integration test (on top of the existing S3A acceptance tests).
I plan to raise an MR for HDDS-10390 early next week.
Regards,
Ivan Andika
> On 4 Oct 2024, at 4:50 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> Hi I know Ozone's acceptance test verifies aws s3 cli against Ozone S3
> Gateway.
>
> Do we test against Hadoop S3A client? Hadoop community is reporting a
> regression due to AWS v2 SDK (*) so I'm wondering if we have the test
> coverage to ensure compatibility.
>
> In addition, do we test or do we plan to test AWS cli v2
>
> * Hadoop 3.4 updated to AWS SDK v2.
Re: Do we test against Hadoop S3A?
Posted by Attila Doroszlai <ad...@apache.org>.
> Do we test against Hadoop S3A client?
We run S3A contract tests from Hadoop using Ozone (running in
docker-compose) as S3 endpoint. Some tests are skipped due to known
issues.
https://github.com/apache/ozone/blob/789fb53f9e07e636134b0bace8a2876dcaa85f09/hadoop-ozone/dist/src/main/compose/common/s3a-test.sh#L95-L100
Most recent run:
https://github.com/apache/ozone/actions/runs/11191727692/job/31115173972
-Attila
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org