[jira] Commented: (HADOOP-3941) Extend FileSystem API to return file-checksums/file-digests

["Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622377#action_12622377 ] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3941:
------------------------------------------------

How about we add the following optional method in the FileSystem API?
{code}
//a new optional method in FileSystem.java
public abstract FileChecksum getFileChecksum(String algorithm, Path p);
{code}
where FileChecksum is a new interface in hadoop.fs package
{code}
interface FileChecksum {
  String getAlgorithm();

  int getLength();

  byte[] getBytes();
}
{code}

> Extend FileSystem API to return file-checksums/file-digests
> -----------------------------------------------------------
>
>                 Key: HADOOP-3941
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3941
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo (Nicholas), SZE
>
> Suppose we have two files in two locations (may be two clusters) and these two files have the same size.  How could we tell whether the content of them are the same?
> Currently, the only way is to read both files and compare the content of them.  This is a very expensive operation if the files are huge.
> So, we would like to extend the FileSystem API to support returning file-checksums/file-digests.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.