You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Yuexin Zhang (JIRA)" <ji...@apache.org> on 2018/07/26 06:51:00 UTC
[jira] [Resolved] (HDFS-13764) [DOC] update flag is not necessary
to avoid verifying checksums
[ https://issues.apache.org/jira/browse/HDFS-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuexin Zhang resolved HDFS-13764.
---------------------------------
Resolution: Won't Fix
it turns out -skipcrccheck needs to be used along with -update:
hadoop distcp -skipcrccheck /tmp1 /tmp2
18/07/25 23:37:06 ERROR tools.DistCp: Invalid arguments:
java.lang.IllegalArgumentException: Skip CRC is valid only with update options
at org.apache.hadoop.tools.DistCpOptions.validate(DistCpOptions.java:691)
> [DOC] update flag is not necessary to avoid verifying checksums
> ---------------------------------------------------------------
>
> Key: HDFS-13764
> URL: https://issues.apache.org/jira/browse/HDFS-13764
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
> Affects Versions: 2.7.0, 2.7.3
> Reporter: Yuexin Zhang
> Priority: Major
> Attachments: HDFS-13764_DOC1.patch
>
>
> We mentioned to use "-update" option to avoid checksum in the following doc:
> [https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Copying_between_encrypted_and_unencrypted_locations]
> {code:java}
> Copying between encrypted and unencrypted locations
> By default, distcp compares checksums provided by the filesystem to verify that the data was successfully copied to the destination. When copying between an unencrypted and encrypted location, the filesystem checksums will not match since the underlying block data is different. In this case, specify the -skipcrccheck and -update distcp flags to avoid verifying checksums.
> {code}
>
> But actually, "-update" option is not necessary, only "-skipcrccheck" is needed. Can we change it to:
>
> {code:java}
> Copying between encrypted and unencrypted locations
> By default, distcp compares checksums provided by the filesystem to verify that the data was successfully copied to the destination. When copying between an unencrypted and encrypted location, the filesystem checksums will not match since the underlying block data is different. In this case, specify the -skipcrccheck flags to avoid verifying checksums.
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org