You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/01/13 06:39:00 UTC
[jira] [Work logged] (HDFS-16422) Fix thread safety of EC decoding during concurrent preads
[ https://issues.apache.org/jira/browse/HDFS-16422?focusedWorklogId=708089&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-708089 ]
ASF GitHub Bot logged work on HDFS-16422:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jan/22 06:38
Start Date: 13/Jan/22 06:38
Worklog Time Spent: 10m
Work Description: cndaimin opened a new pull request #3881:
URL: https://github.com/apache/hadoop/pull/3881
Reading data on an erasure-coded file with missing replicas(internal block of block group) will cause online reconstruction: read `dataUnits` part of data and decode them into the target missing data. Each `DFSStripedInputStream` object has a `RawErasureDecoder` object, and when we doing pread concurrently, `RawErasureDecoder.decode` will be invoked concurrently too. `RawErasureDecoder.decode` is not thread safe, as a result of that we get wrong data from pread occasionally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 708089)
Remaining Estimate: 0h
Time Spent: 10m
> Fix thread safety of EC decoding during concurrent preads
> ---------------------------------------------------------
>
> Key: HDFS-16422
> URL: https://issues.apache.org/jira/browse/HDFS-16422
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: dfsclient, ec, erasure-coding
> Affects Versions: 3.3.0, 3.3.1
> Reporter: daimin
> Assignee: daimin
> Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Reading data on an erasure-coded file with missing replicas(internal block of block group) will cause online reconstruction: read dataUnits part of data and decode them into the target missing data. Each DFSStripedInputStream object has a RawErasureDecoder object, and when we doing pread concurrently, RawErasureDecoder.decode will be invoked concurrently too. RawErasureDecoder.decode is not thread safe, as a result of that we get wrong data from pread occasionally.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org