You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2020/01/02 15:19:00 UTC
[jira] [Commented] (HADOOP-16762) Add support for
Filesystem#getFileChecksum in ABFS driver
[ https://issues.apache.org/jira/browse/HADOOP-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17006854#comment-17006854 ]
Steve Loughran commented on HADOOP-16762:
-----------------------------------------
I support this, with one warning: once a far system starts returning a checksum, unless you tell distcp to to skip the check, it will be unhappy if the crcs don't match (well, it will think the file always needs copying from HDFS or other source). I got hit by this when I modified S3a to return the etag, so I had to make that optional.
Oh, and the CDP docs always use -skipcrccheck in their examples of backups to object stores, though HADOOP-16756 implies there's more to do
> Add support for Filesystem#getFileChecksum in ABFS driver
> ---------------------------------------------------------
>
> Key: HADOOP-16762
> URL: https://issues.apache.org/jira/browse/HADOOP-16762
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Virajith Jalaparti
> Priority: Major
>
> Currently, ABFS driver does not support Filesystem#getFileChecksum even though the underlying ADLS REST API does.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org