You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Robert Chansler (JIRA)" <ji...@apache.org> on 2008/10/22 01:24:44 UTC
[jira] Updated: (HADOOP-3328) DFS write pipeline : only the last
datanode needs to verify checksum
[ https://issues.apache.org/jira/browse/HADOOP-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3328:
------------------------------------
Release Note: (was: When client is writing data to DFS, only the lastdatanode in the pipeline needs to verify the checksum. Saves around 30% CPU on intermediate datanodes. )
> DFS write pipeline : only the last datanode needs to verify checksum
> --------------------------------------------------------------------
>
> Key: HADOOP-3328
> URL: https://issues.apache.org/jira/browse/HADOOP-3328
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.19.0
>
> Attachments: HADOOP-3328.patch, HADOOP-3328.patch
>
>
> Currently all the datanodes in DFS write pipeline verify checksum. Since the current protocol includes acks from the datanodes, an ack from the last node could also serve as verification that checksum ok. In that sense, only the last datanode needs to verify checksum. Based on [this comment|http://issues.apache.org/jira/browse/HADOOP-1702?focusedCommentId=12575553#action_12575553] from HADOOP-1702, CPU consumption might go down by another 25-30% (4/14) after HADOOP-1702.
> Also this would make it easier to use transferTo() and transferFrom() on intermediate datanodes since they don't need to look at the data.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.