You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ludun (Jira)" <ji...@apache.org> on 2020/06/23 09:01:00 UTC
[jira] [Updated] (HDFS-15431) Can not read a opening file after
NameNode failover if pipeline recover occuered
[ https://issues.apache.org/jira/browse/HDFS-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ludun updated HDFS-15431:
-------------------------
Description:
a file with two replications and keep it opening.
first it writes to DN1 and DN2.
{code}
2020-06-23 14:22:51,379 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-7e434b35-0b10-44fa-9d3b-c3c938f1724d,DISK]] | DataStreamer.java:1757
{code}
after DN2 restart, it writes to DN1 and DN3,
{code}
2020-06-23 14:24:04,559 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK]] | DataStreamer.java:1757
{code}
after DN1 restart. it writes to DN3 and DN4.
{code}
2020-06-23 14:25:21,340 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK], DatanodeInfoWithStorage[DN4:25009,DS-5fbb2232-e7c8-4186-8eb9-87a6aff86cef,DISK]] | DataStreamer.java:1757
{code}
restart Active NameNode. then try to get the file.
NameNode return locatedblocks with DN1 and DN2. Can not obtain block Exception occurred.
{code}
20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
fileLength=0
underConstruction=true
blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
isLastBlockComplete=false}
{code}
was:
a file with two replications and keep it opening.
first it writes to DN1 and DN2.
after DN2 restart, it writes to DN1 and DN3,
after DN1 restart. it writes to DN3 and DN4.
restart Active NameNode. then try to get the file.
NameNode return locatedblocks with DN1 and DN2. Can not obtain block Exception occurred.
{code}
20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
fileLength=0
underConstruction=true
blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
isLastBlockComplete=false}
{code}
> Can not read a opening file after NameNode failover if pipeline recover occuered
> --------------------------------------------------------------------------------
>
> Key: HDFS-15431
> URL: https://issues.apache.org/jira/browse/HDFS-15431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: ludun
> Priority: Major
>
> a file with two replications and keep it opening.
> first it writes to DN1 and DN2.
> {code}
> 2020-06-23 14:22:51,379 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-7e434b35-0b10-44fa-9d3b-c3c938f1724d,DISK]] | DataStreamer.java:1757
> {code}
> after DN2 restart, it writes to DN1 and DN3,
> {code}
> 2020-06-23 14:24:04,559 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK]] | DataStreamer.java:1757
> {code}
> after DN1 restart. it writes to DN3 and DN4.
> {code}
> 2020-06-23 14:25:21,340 | DEBUG | pipeline = [DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK], DatanodeInfoWithStorage[DN4:25009,DS-5fbb2232-e7c8-4186-8eb9-87a6aff86cef,DISK]] | DataStreamer.java:1757
> {code}
> restart Active NameNode. then try to get the file.
> NameNode return locatedblocks with DN1 and DN2. Can not obtain block Exception occurred.
> {code}
> 20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
> fileLength=0
> underConstruction=true
> blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
> lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796; getBlockSize()=53; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK], DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
> isLastBlockComplete=false}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org