You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/04/15 09:32:00 UTC
[jira] [Work logged] (HDFS-16544) EC decoding failed due to invalid buffer
[ https://issues.apache.org/jira/browse/HDFS-16544?focusedWorklogId=757371&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757371 ]
ASF GitHub Bot logged work on HDFS-16544:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Apr/22 09:31
Start Date: 15/Apr/22 09:31
Worklog Time Spent: 10m
Work Description: liubingxing opened a new pull request, #4179:
URL: https://github.com/apache/hadoop/pull/4179
In [HDFS-16538](http://https//issues.apache.org/jira/browse/HDFS-16538) , we found an EC file decoding bug if more than one data block read failed.
Currently, we found another bug trigger by #StatefulStripeReader.decode.
If we read an EC file which **length more than one stripe**, and this file have **one data block** and **the first parity block** corrupted, this error will happen.
```java
org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not allowing null at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132)
at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
at org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435)
at org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94)
at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392)
at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315)
at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918)
```
Issue Time Tracking
-------------------
Worklog Id: (was: 757371)
Remaining Estimate: 0h
Time Spent: 10m
> EC decoding failed due to invalid buffer
> ----------------------------------------
>
> Key: HDFS-16544
> URL: https://issues.apache.org/jira/browse/HDFS-16544
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: qinyuren
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In [HDFS-16538|http://https//issues.apache.org/jira/browse/HDFS-16538] , we found an EC file decoding bug if more than one data block read failed.
> Currently, we found another bug trigger by #StatefulStripeReader.decode.
> If we read an EC file which {*}length more than one stripe{*}, and this file have *one data block* and *the first parity block* corrupted, this error will happen.
> {code:java}
> org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not allowing null at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132)
> at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
> at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
> at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
> at org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435)
> at org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94)
> at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392)
> at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315)
> at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918) {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org