You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Vinayakumar B (JIRA)" <ji...@apache.org> on 2019/06/20 05:00:06 UTC
[jira] [Commented] (HDFS-14581) Appending to EC files crashes
NameNode
[ https://issues.apache.org/jira/browse/HDFS-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868262#comment-16868262 ]
Vinayakumar B commented on HDFS-14581:
--------------------------------------
Fix looks good to me.
Once the following message change is done, it will be good to go.
{quote} Regarding the exception message, may be an added line of suggestion to use the flag as of now can be done.
{quote}
> Appending to EC files crashes NameNode
> --------------------------------------
>
> Key: HDFS-14581
> URL: https://issues.apache.org/jira/browse/HDFS-14581
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.3.0
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
> Priority: Critical
> Attachments: HDFS-14581.001.patch
>
>
> *org.apache.hadoop.hdfs.server.namenode.FSDirAppendOp.prepareFileForAppend(..)@189*
> {noformat}
> file.recordModification(iip.getLatestSnapshotId());
> file.toUnderConstruction(leaseHolder, clientMachine);
> fsn.getLeaseManager().addLease(
> file.getFileUnderConstructionFeature().getClientName(), file.getId());
> LocatedBlock ret = null;
> if (!newBlock) {
> if (file.isStriped()) {
> throw new UnsupportedOperationException(
> "Append on EC file without new block is not supported.");
> }{noformat}
> In this code "UnsupportedOperationException" exception thows after marking file underConstruction. In this case file is opened without any "Open" editlogs, after some time lease manager close this file and add close edit log.
> When SBN tail this edit log, it will fail with this exception.
> {noformat}
> 2019-06-13 19:17:51,513 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader: Encountered exception on operation CloseOp [length=0, inodeId=0, path=/ECtest/********, replication=1, mtime=1560261947480, atime=1560258249117, blockSize=134217728, blocks=[blk_-9223372036854775792_1005], permissions=root:hadoop:rw-r--r--, aclEntries=null, clientName=, clientMachine=, overwrite=false, storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_CLOSE, txid=1363]
> java.io.IOException: File is not under construction: /ECtest/container-executor
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:504)
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:286)
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:181)
> at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:924)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:329)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:410)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org