You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/05/14 20:43:00 UTC
[jira] [Commented] (HDFS-17528) FsImageValidation: set txid when saving a new image
[ https://issues.apache.org/jira/browse/HDFS-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846440#comment-17846440 ]
ASF GitHub Bot commented on HDFS-17528:
---------------------------------------
szetszwo opened a new pull request, #6828:
URL: https://github.com/apache/hadoop/pull/6828
### Description of PR
- When the fsimage is specified as a file and the FsImageValidation tool saves a new image (for removing inaccessible inodes), the txid is not set. Then, the resulted image will have 0 as its txid.
- When the fsimage is specified as a directory, the txid is set. However, it will get NPE since NameNode metrics is uninitialized (although the metrics is not used by FsImageValidation).
### How was this patch tested?
Tested manually
- before: the output file is `fsimage.ckpt_0000000000000000000` (i.e. txid is 0)
> 2024-05-14 13:37:27,531 [main] INFO namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:save(732)) - Saving image file /Users/szetszwo/hadoop/fsimage/current/newFsImage5968764763996132609/current/fsimage.ckpt_0000000000000000000 using no compression
> 2024-05-14 13:37:30,522 [main] INFO namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:save(736)) - Image file /Users/szetszwo/hadoop/fsimage/current/newFsImage5968764763996132609/current/fsimage.ckpt_0000000000000000000 of size 200392059 bytes saved in 2 seconds .
- after: the output file is `fsimage.ckpt_0000000023945925442` with correct txid
> 2024-05-14 13:38:32,414 [main] INFO namenode.FSImage (FSImage.java:save(1223)) - save fsimage with txid=23945925442 to /Users/szetszwo/hadoop/fsimage/current/newFsImage4409944859316006440
> 2024-05-14 13:38:32,436 [main] INFO namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:save(732)) - Saving image file /Users/szetszwo/hadoop/fsimage/current/newFsImage4409944859316006440/current/fsimage.ckpt_0000000023945925442 using no compression
> 2024-05-14 13:38:35,437 [main] INFO namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:save(736)) - Image file /Users/szetszwo/hadoop/fsimage/current/newFsImage4409944859316006440/current/fsimage.ckpt_0000000023945925442 of size 200392062 bytes saved in 3 seconds .
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [NA] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [NA] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [NA] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
> FsImageValidation: set txid when saving a new image
> ---------------------------------------------------
>
> Key: HDFS-17528
> URL: https://issues.apache.org/jira/browse/HDFS-17528
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
>
> - When the fsimage is specified as a file and the FsImageValidation tool saves a new image (for removing inaccessible inodes), the txid is not set. Then, the resulted image will have 0 as its txid.
> - When the fsimage is specified as a directory, the txid is set. However, it will get NPE since NameNode metrics is uninitialized (although the metrics is not used by FsImageValidation).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org