You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Rashmi Vinayak (JIRA)" <ji...@apache.org> on 2016/02/10 06:35:18 UTC

[jira] [Commented] (HADOOP-11828) Implement the Hitchhiker erasure coding algorithm

    [ https://issues.apache.org/jira/browse/HADOOP-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140359#comment-15140359 ] 

Rashmi Vinayak commented on HADOOP-11828:
-----------------------------------------

Hi [~jack_liuquan], [~drankye], [~zhz],

I am super excited to see this being resolved! Thank you all for the efforts that you put in. I agree with [~zhz] that it would be good to get some performance results comparing RS and Hitchhiker based on the new implementation. This would guide enterprises who are considering using erasure coding, and thus leading to a greater impact from this effort and HDFS-EC in general as they will come to know about this more efficient EC option. 

> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
>                 Key: HADOOP-11828
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11828
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0
>            Reporter: Zhe Zhang
>            Assignee: jack liuquan
>             Fix For: 3.0.0
>
>         Attachments: 7715-hitchhikerXOR-v2-testcode.patch, 7715-hitchhikerXOR-v2.patch, HADOOP-11828-hitchhikerXOR-V3.patch, HADOOP-11828-hitchhikerXOR-V4.patch, HADOOP-11828-hitchhikerXOR-V5.patch, HADOOP-11828-hitchhikerXOR-V6.patch, HADOOP-11828-hitchhikerXOR-V7.patch, HADOOP-11828-v8.patch, HDFS-7715-hhxor-decoder.patch, HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker | http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is a new erasure coding algorithm developed as a research project at UC Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45% during data reconstruction while retaining the same storage capacity and failure tolerance capability as RS codes. This JIRA aims to introduce Hitchhiker to the HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)