[jira] [Commented] (HDFS-8150) Make getFileChecksum fail for blocks under construction

["Akira AJISAKA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-8150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533744#comment-14533744 ] 

Akira AJISAKA commented on HDFS-8150:
-------------------------------------

Thanks [~andreina] for taking this issue. I agree with you that DFSClient should throw exception if the file is under construction. Some comments:
{code}
+   if (blockLocations.isUnderConstruction()) {
+     throw new IOException("Fail to get block MD5, since file " + src + " is under construction ");
+   }
{code}
1. Would you throw the exception when refreching block locations as well?
2. For "block MD5", I'm thinking "checksum" is sufficient. We can get block MD5 checksum from finalized blocks even if the file is under construction.
3. nit: Would you remove unnecessarily whitespace after "construction"?
4. nit: The line is longer than 80 characters.

{code}
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertThat;
+import static org.junit.Assert.*;
{code}
5. nit: Would you please avoid using * for import?

> Make getFileChecksum fail for blocks under construction
> -------------------------------------------------------
>
>                 Key: HDFS-8150
>                 URL: https://issues.apache.org/jira/browse/HDFS-8150
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Assignee: J.Andreina
>            Priority: Critical
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-8150.1.patch, HDFS-8150.2.patch
>
>
> We have seen the cases of validating data copy using checksum then the content of target changing. It turns out the target wasn't closed successfully, so it was still under-construction.  One hour later, a lease recovery kicked in and truncated the block.
> Although this can be prevented in many ways, if there is no valid use case for getting file checksum from under-construction blocks, can it be disabled?  E.g. Datanode can throw an exception if the replica is not in the finalized state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)