You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Tyler Hale (JIRA)" <ji...@apache.org> on 2015/06/23 22:29:45 UTC

[jira] [Created] (MAPREDUCE-6414) Distcp command very slow to enumerate files needing

Tyler Hale created MAPREDUCE-6414:
-------------------------------------

             Summary: Distcp command very slow to enumerate files needing
                 Key: MAPREDUCE-6414
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6414
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: distcp
    Affects Versions: 2.5.0
         Environment: RHEL 6.5
            Reporter: Tyler Hale


When copying large amounts of data using distcp utility (100's of TBs), the distcp utility takes a large time to enumerate all of the files that have changed.  In my system, this corresponds to 14-16 hours before the actual copying of data begins. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)