[jira] Updated: (MAPREDUCE-2275) RaidNode should monitor and fix blocks that violate RAID block placement

["Scott Chen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Chen updated MAPREDUCE-2275:
----------------------------------

    Attachment: MAPREDUCE-2275.txt

> RaidNode should monitor and fix blocks that violate RAID block placement 
> -------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2275
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2275
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/raid
>            Reporter: Ramkumar Vadali
>            Assignee: Ramkumar Vadali
>         Attachments: MAPREDUCE-2275.txt
>
>
> When files are RAIDed, it is important to keep blocks in each RAID stripe and the corresponding parity blocks on as many different machines as possible. This ensures minimal probability of data loss when data nodes go dead.
> BlockPlacementPolicyRaid ensures that parity blocks are not located on the same machines as the source blocks. But source blocks placement is not controlled directly in this manner. Instead, source blocks are allowed to be created using the default policy. After a source file is RAIDed, its replication is increased, and then decreased. BlockPlacementPolicyRaid then tries to keep the source blocks well-located when excess blocks are deleted. This is not guaranteed to ensure the correct block placement for RAID.
> Also, if blocks are moved around by the balancer, the block placement could be violated.
> We need periodic monitoring of block placement of RAIDed files and the corresponding parity blocks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.