You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (JIRA)" <ji...@apache.org> on 2011/07/09 22:25:59 UTC

[jira] [Commented] (MAPREDUCE-2406) Failed validate copy in distcp

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13062613#comment-13062613 ] 

Harsh J commented on MAPREDUCE-2406:
------------------------------------

Looking at validateCopy in trunk tells me that it does not seem to check anything beyond the crc (if available, and if asked for) and the length of whole files. That doesn't seem to be a cause for a failure due to differing block sizes here?

> Failed validate copy in distcp
> ------------------------------
>
>                 Key: MAPREDUCE-2406
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2406
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Rosie Li
>            Priority: Minor
>
> Each time the distcp is done, {{validateCopy(srcstat, absdst)}} will be called. 
> When doing distcp, if the -pb(preserve block size) is not set, the dst will use the default block size. However, if the src file use block size other than the default block size, and -pb is not set, after copying, the src and dst will have different block size. It will not pass the validateCopy check in this case.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira