You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2021/02/19 16:37:00 UTC
[jira] [Updated] (HADOOP-17483) magic committer to be enabled for
all S3 buckets
[ https://issues.apache.org/jira/browse/HADOOP-17483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-17483:
------------------------------------
Description:
* core-default.xml updated so that fs.s3a.committer.magic.enabled = true
* CommitConstants updated to match
* All tests which previously enabled the magic committer now rely on
default settings. This helps make sure it is enabled.
* Docs cover the switch, mention its enabled and explain why you may
want to disable it.
Note: this doesn't switch to using the committer -it just enables the path
rewriting magic which it depends on.
was:
now that S3 is consistent, there is no need to disable the magic committer for safety.
remove option to enable magic committer (fs.s3a.committer.magic.enabled) and the associated checks/probes through the code.
May want to retain the constants and probes just for completeness/API/CLI consistency.
> magic committer to be enabled for all S3 buckets
> ------------------------------------------------
>
> Key: HADOOP-17483
> URL: https://issues.apache.org/jira/browse/HADOOP-17483
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> * core-default.xml updated so that fs.s3a.committer.magic.enabled = true
> * CommitConstants updated to match
> * All tests which previously enabled the magic committer now rely on
> default settings. This helps make sure it is enabled.
> * Docs cover the switch, mention its enabled and explain why you may
> want to disable it.
> Note: this doesn't switch to using the committer -it just enables the path
> rewriting magic which it depends on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org