[jira] [Commented] (HADOOP-15273) distcp error message on checksum mismatch is misleading when checksum protocol itself is different

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380472#comment-16380472 ] 

Steve Loughran commented on HADOOP-15273:
-----------------------------------------

really the checksum classes need to be examined, and if different, go to a message like

bq. The two filesystems use different checksum protocols; you need to disable checksum validation with -skipCrc.

Best option: use getAlgorithm() to compare algorithm strings, and if different, explicitly declare that as the cause

> distcp error message on checksum mismatch is misleading when checksum protocol itself is different
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-15273
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15273
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: tools/distcp
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Minor
>
> When using distcp without {{-skipCRC}} . If there's a checksum mismatch between src and dest store types (e.g hdfs to s3), then the error message will talk about blocksize, even when its the underlying checksum protocol itself which is the cause for failure
> bq. Source and target differ in block-size. Use -pb to preserve block-sizes during copy. Alternatively, skip checksum-checks altogether, using -skipCrc. (NOTE: By skipping checksums, one runs the risk of masking data-corruption during file-transfer.)
> IF the checksum types are fundamentally different, the error message should say so



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org