You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/03/06 10:22:00 UTC
[jira] [Commented] (HDFS-17401) EC: Excess internal block may not be able to be deleted correctly when it's stored in fallback storage
[ https://issues.apache.org/jira/browse/HDFS-17401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823952#comment-17823952 ]
ASF GitHub Bot commented on HDFS-17401:
---------------------------------------
zhangshuyan0 commented on code in PR #6597:
URL: https://github.com/apache/hadoop/pull/6597#discussion_r1514212224
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestReconstructStripedBlocks.java:
##########
@@ -27,13 +27,7 @@
import org.apache.hadoop.hdfs.MiniDFSCluster;
import org.apache.hadoop.hdfs.StripedFileTestUtil;
import org.apache.hadoop.hdfs.client.HdfsDataInputStream;
-import org.apache.hadoop.hdfs.protocol.Block;
-import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
-import org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy;
-import org.apache.hadoop.hdfs.protocol.LocatedBlock;
-import org.apache.hadoop.hdfs.protocol.LocatedBlocks;
-import org.apache.hadoop.hdfs.protocol.LocatedStripedBlock;
-import org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies;
+import org.apache.hadoop.hdfs.protocol.*;
Review Comment:
Please do not use * in `import`.
> EC: Excess internal block may not be able to be deleted correctly when it's stored in fallback storage
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17401
> URL: https://issues.apache.org/jira/browse/HDFS-17401
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.3.6
> Reporter: Ruinan Gu
> Assignee: Ruinan Gu
> Priority: Major
> Labels: pull-request-available
>
> Excess internal block can't be deleted correctly when it's stored in fallback storage.
> Simple case:
> EC-RS-6-3-1024k file is stored using ALL_SSD storage policy(SSD is default storage type and DISK is fallback storage type), if the block group is as follows
> [0(SSD), 0(SSD), 1(SSD), 2(SSD), 3(SSD), 4(SSD), 5(SSD), 6(SSD), 7(SSD), 8(DISK)]
> The are two index 0 internal block and one of them should be chosen to delete.But the current implement chooses the index 0 internal blocks as candidates but DISK as exess storage type.As a result, the exess storage type(DISK) can not correspond to the exess internal blocks' storage type(SSD) correctly, and the exess internal block can not be deleted correctly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org