You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/08/19 03:48:00 UTC
[jira] [Work logged] (HADOOP-17851) Support user specified content
encoding for S3A
[ https://issues.apache.org/jira/browse/HADOOP-17851?focusedWorklogId=639690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-639690 ]
ASF GitHub Bot logged work on HADOOP-17851:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 19/Aug/21 03:47
Start Date: 19/Aug/21 03:47
Worklog Time Spent: 10m
Work Description: holdenk opened a new pull request #3312:
URL: https://github.com/apache/hadoop/pull/3312
### Description of PR
Add support for user specified content encoding for S3A.
### How was this patch tested?
Run existing unit tests, new integration test, and run S3A integration tests in us-east-2. Added some small changes to S3 integration tests to skip encryption tests when key is null or alg is null (I believe this matches the original intention given the use of `StringUtils.isBlank` for skipping the test).
### For code changes:
- [ X ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ X ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?:
(us-east-2 ran with `holden@a61a514145cb:~/hadoop/hadoop-tools/hadoop-aws$ mvn clean verify -Dnon-auth -Ds3guard` inside of the docker dev instance)
Notable failures, which existed without the change are `org.apache.hadoop.fs.s3a.ITestS3AMiscOperations` (namely ` checksums of empty files expected:<etag: "5e3037f6b23642c732f1b705db9dcf17"> but was:<etag: "e5321d4514ae69ccf36995e8e1ac846f">`) . I can dig into this more if this isn't expected/flaky.
- [ N/A ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ N/A ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 639690)
Remaining Estimate: 0h
Time Spent: 10m
> Support user specified content encoding for S3A
> -----------------------------------------------
>
> Key: HADOOP-17851
> URL: https://issues.apache.org/jira/browse/HADOOP-17851
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hadoop-thirdparty
> Reporter: Holden Karau
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> User-specified object content-encoding (part of the object metadata) is important for allowing compressed files to be processed in the AWS ecosystem. We should allow the user to specify the content encoding of the files being written.
> metadata can not be changed after a file is written without a rewrite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org