[jira] [Updated] (MAPREDUCE-5809) Enhance distcp to support preserving HDFS ACLs.

["Chris Nauroth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Nauroth updated MAPREDUCE-5809:
-------------------------------------

    Attachment: MAPREDUCE-5809.1.patch

The attached patch introduces the "distcp -pa" command line argument for preserving ACLs.  I found that it helped the implementation to reuse some code currently in HDFS, so I've done some refactoring.  If this gets +1, then I'll split it into separate patches for HADOOP and HDFS.

# {{AclUtil}}: This is a new utility class containing methods that previously resided in HDFS or privately in Hadoop Common.  It's annotated {{LimitedPrivate}} for HDFS and MapReduce.  There is no new logic here, just moving the methods around.
# {{ScopedAclEntries}}: This whole class has moved from HDFS to Common with addition of the {{LimitedPrivate}} annotation.
# {{AclCommands}}: The implementation of the ls shell command is simplified by using the new utility code.
# {{CopyListing}}: Checks if source file system supports ACLs before attempting to run the distcp job (fail fast).  Renamed {{checkForDuplicates}} to {{validateFinalListing}} since the validation is now more than just checking for duplicates.
# {{DistCp}}: Checks if target file system supports ACLs.
# {{DistCpOptionSwitch}}: Documented meaning of -pa and also made it clear which attributes get preserved when passing -p with no additional flags.
# {{DistCpUtils}}: Preserve ACLs if requested.
# {{TestDistCpWithAcls}}: New test suite that runs distcp with -pa and asserts that ACLs and permission bits are the same at the destination.  Also tests the fail-fast behavior when ACLs are not enabled in the NameNode or ACLs are unimplemented in the file system.

The one thing we can't cover in our automated tests here is the case of attempting distcp with -pa where the NameNode is pre-2.4.0.  I'll check that with a manual test.

> Enhance distcp to support preserving HDFS ACLs.
> -----------------------------------------------
>
>                 Key: MAPREDUCE-5809
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5809
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: distcp
>    Affects Versions: 2.4.0
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: MAPREDUCE-5809.1.patch
>
>
> This issue tracks enhancing distcp to add a new command-line argument for preserving HDFS ACLs from the source at the copy destination.



--
This message was sent by Atlassian JIRA
(v6.2#6252)