[jira] Commented: (HDFS-808) Implement something like PAR2 support?

["Eli Collins (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785999#action_12785999 ] 

Eli Collins commented on HDFS-808:
----------------------------------

You could use the parity blocks in the erasure encoding layer (HDFS-503) to implement the parity blocks in PAR2, though given that HDFS has multiple replicas why are parity blocks better than the current recovery mechanism (which just uses a non-corrupt block rather than reconstructing the corrupt block from parity)?

> Implement something like PAR2 support?
> --------------------------------------
>
>                 Key: HDFS-808
>                 URL: https://issues.apache.org/jira/browse/HDFS-808
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Allen Wittenauer
>            Priority: Minor
>
> We really need an Idea issue type, because I'm not sure if this is really viable. :)  Just sort of thinking "out loud".
> I was thinking about how file recovery works on services like Usenet to fix data corruption when chunks of files are missing.  I wonder how hard it would be to implement something like PAR2 [ http://en.wikipedia.org/wiki/Parchive ] automatically for large files.  We'd have the advantage of being able to do it in binary of course and could likely hide the details within HDFS itself.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.