Erasure Coding Policies

[Nazerke S <sn...@gmail.com>]

Hi,


What is the difference between  LEGACY-RS-6-3-1024k and RS-6-3-1024k EC
policies?

Re: Erasure Coding Policies

[Rakesh Radhakrishnan <ra...@apache.org>]

Hi,

RS Legacy is pure Java based implementation. Probably you can look at the
encoding/decoding logic at github repo
https://github.com/Jerry-Xin/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RSRawEncoderLegacy.java

https://github.com/Jerry-Xin/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/RSRawDecoderLegacy.java

RS codec, uses a native implementation which leverages Intel ISA-L library
to improve the performance of codec. Please go through below links to get
more details on ISA-L part.
https://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Enable_Intel_ISA-L

https://blog.cloudera.com/blog/2019/06/hdfs-erasure-coding-in-production/

Rakesh

On Mon, Jul 1, 2019 at 5:56 PM Nazerke S <sn...@gmail.com> wrote:

> Hi,
>
>
> What is the difference between  LEGACY-RS-6-3-1024k and RS-6-3-1024k EC
> policies?
>
>
>