You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Kai Zheng (JIRA)" <ji...@apache.org> on 2017/09/21 07:02:01 UTC
[jira] [Resolved] (HDFS-7337) Configurable and pluggable erasure
codec and policy
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kai Zheng resolved HDFS-7337.
-----------------------------
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
Release Note:
This allows users to:
* develop and plugin their own erasure codec and coders. The plugin will be loaded automatically from hadoop jars, the corresponding codec and coder will be registered for runtime use.
* define their own erasure coding policies thru an xml file and CLI command. The added policies will be persisted into fsimage.
> Configurable and pluggable erasure codec and policy
> ---------------------------------------------------
>
> Key: HDFS-7337
> URL: https://issues.apache.org/jira/browse/HDFS-7337
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: erasure-coding
> Reporter: Zhe Zhang
> Priority: Critical
> Labels: hdfs-ec-3.0-nice-to-have
> Fix For: 3.0.0-beta1
>
> Attachments: HDFS-7337-prototype-v1.patch, HDFS-7337-prototype-v2.zip, HDFS-7337-prototype-v3.zip, PluggableErasureCodec.pdf, PluggableErasureCodec-v2.pdf, PluggableErasureCodec-v3.pdf, PluggableErasureCodec v4.pdf
>
>
> According to HDFS-7285 and the design, this considers to support multiple Erasure Codecs via pluggable approach. It allows to define and configure multiple codec schemas with different coding algorithms and parameters. The resultant codec schemas can be utilized and specified via command tool for different file folders. While design and implement such pluggable framework, it’s also to implement a concrete codec by default (Reed Solomon) to prove the framework is useful and workable. Separate JIRA could be opened for the RS codec implementation.
> Note HDFS-7353 will focus on the very low level codec API and implementation to make concrete vendor libraries transparent to the upper layer. This JIRA focuses on high level stuffs that interact with configuration, schema and etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org