You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2017/03/29 17:23:41 UTC

[jira] [Created] (HADOOP-14254) Add a Distcp option to preserve Erasure Coding attributes

Wei-Chiu Chuang created HADOOP-14254:
----------------------------------------

             Summary: Add a Distcp option to preserve Erasure Coding attributes
                 Key: HADOOP-14254
                 URL: https://issues.apache.org/jira/browse/HADOOP-14254
             Project: Hadoop Common
          Issue Type: Improvement
          Components: tools/distcp
    Affects Versions: 3.0.0-alpha3
            Reporter: Wei-Chiu Chuang


Currently Distcp does not preserve the erasure coding attributes properly. I propose we add a "-pe" switch to ensure erasure coded files at source are copied as erasure coded files at destination.

For example, if the src cluster has the following directories and files that are copied to dest cluster
hdfs://src/ root directory is replicated
hdfs://src/foo erasure code enabled directory
hdfs://src/foo/bar erasure coded file

after distcp, hdfs://dest/foo and hdfs://dest/foo/bar will not be erasure coded. 

It may be useful to add such capability. One potential use is for disaster recovery. The other use is for out-of-place cluster upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org