You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2019/08/13 00:27:00 UTC
[jira] [Resolved] (HDFS-14148) HDFS OIV ReverseXML SnapshotSection
parser throws exception when there are more than one snapshottable
directory
[ https://issues.apache.org/jira/browse/HDFS-14148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei-Chiu Chuang resolved HDFS-14148.
------------------------------------
Resolution: Fixed
Fix Version/s: 3.3.0
PR was merged. Thanks [~smeng] for reporting the issue and fixing the bug!
> HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory
> ----------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-14148
> URL: https://issues.apache.org/jira/browse/HDFS-14148
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 3.0.0
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14148.test.001.patch, fsimage_0000000000000000024, fsimage_0000000000000000441
>
>
> The current HDFS OIV tool doesn't seem to support snapshot well when reversing XML back to binary.
> {code:bash}
> $ hdfs oiv -i fsimage_0000000000000026542.xml -o reverse.bin -p ReverseXML
> OfflineImageReconstructor failed: Found unknown XML keys in <dir>: dir
> java.io.IOException: Found unknown XML keys in <dir>: dir
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$Node.verifyNoRemainingKeys(OfflineImageReconstructor.java:324)
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$SnapshotSectionProcessor.process(OfflineImageReconstructor.java:1357)
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processXml(OfflineImageReconstructor.java:1785)
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(OfflineImageReconstructor.java:1840)
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.run(OfflineImageViewerPB.java:193)
> at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.main(OfflineImageViewerPB.java:136)
> $ grep -n "<dir>" fsimage_0000000000000026542.xml
> 228:<INodeReferenceSection></INodeReferenceSection><SnapshotSection><snapshotCounter>2</snapshotCounter><numSnapshots>2</numSnapshots><snapshottableDir><dir>20492</dir><dir>20495</dir></snapshottableDir>
> {code}
> This is also reproduced on latest trunk fba222a85603d6321419aa37bcc48d276dd6c4a6
> This problem can be easily reproduced when there are at least TWO snapshot-enabled directories. Apply HDFS-14148.test.001.patch and run *TestOfflineImageViewer#testReverseXmlRoundTrip()* to see my point.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org