You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "WangYuanben (Jira)" <ji...@apache.org> on 2024/11/13 14:46:00 UTC
[jira] [Updated] (HDFS-17663) File may be corrupted when using high-data-block EC policy with ISAL.
[ https://issues.apache.org/jira/browse/HDFS-17663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
WangYuanben updated HDFS-17663:
-------------------------------
Attachment: image-2024-11-13-22-45-37-445.png
> File may be corrupted when using high-data-block EC policy with ISAL.
> ---------------------------------------------------------------------
>
> Key: HDFS-17663
> URL: https://issues.apache.org/jira/browse/HDFS-17663
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ec, erasure-coding, native
> Reporter: WangYuanben
> Priority: Major
> Attachments: 12-4-test.png, 15-1-test.png, image-2024-11-13-22-45-37-445.png
>
>
> When I define and use the RS-14-2-1024k/RS-13-3-1024k/RS-12-4-1024k EC policy, I find that files are corrupted when decoded with ISAL. The files can be retrieved and have the same size as the original, but their MD5 checksums differ.
> To verify that the encoding is correct, I perform a cross-validation: I encode the file both with and without ISAL, then decode it with and without ISAL. This results in four files, and only the two files decoded with ISAL have different MD5 checksums compared to the original. The picture shows the result:
> !12-4-test.png|width=550,height=333!
> For RS-15-1-1024k, there are issues with both encoding and decoding. Only the file encoded and decoded without ISAL has the same MD5 checksum as the original. The picture shows the result:
> !15-1-test.png|width=550,height=333!
> The test files decoded are as below:
> !files.png|width=500,height=333!
>
> Maybe we should do something to solve it ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org