You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/12/31 07:03:41 UTC
[GitHub] [hadoop] haiyang1987 opened a new pull request, #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
haiyang1987 opened a new pull request, #5264:
URL: https://github.com/apache/hadoop/pull/5264
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
[HDFS-16879](https://issues.apache.org/jira/browse/HDFS-16879)
For the block of the ec file run hdfs fsck -blockId xxx can add shows number of redundant internal block replicas.
### For code changes:
for example: the current blockgroup has 10 live replicas, and it will show there are 9 live replicas.
actually, there is a live replica that should be in the redundant state, can add shows "No. of redundant Replica: 1"
```
hdfs fsck -blockId blk_-xxx
Block Id: blk_-xxx
Block belongs to: /ec/file1
No. of Expected Replica: 9
No. of live Replica: 9
No. of excess Replica: 0
No. of stale Replica: 0
No. of decommissioned Replica: 0
No. of decommissioning Replica: 0
No. of corrupted Replica: 0
Block replica on datanode/rack: ip-xxx1 is HEALTHY
Block replica on datanode/rack: ip-xxx2 is HEALTHY
Block replica on datanode/rack: ip-xxx3 is HEALTHY
Block replica on datanode/rack: ip-xxx4 is HEALTHY
Block replica on datanode/rack: ip-xxx5 is HEALTHY
Block replica on datanode/rack: ip-xxx6 is HEALTHY
Block replica on datanode/rack: ip-xxx7 is HEALTHY
Block replica on datanode/rack: ip-xxx8 is HEALTHY
Block replica on datanode/rack: ip-xxx9 is HEALTHY
Block replica on datanode/rack: ip-xxx10 is HEALTHY
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1369333171
Hi @ZanderXu @tomscut @tasanuma please help me review this pr when you are available, Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1369447026
Get, Thanks @dineshchitlangia help review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ZanderXu merged pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
ZanderXu merged PR #5264:
URL: https://github.com/apache/hadoop/pull/5264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1370454477
Thanks @ZanderXu @dineshchitlangia help review and merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ZanderXu commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
ZanderXu commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1370454618
The failed UT `hadoop.hdfs.TestLeaseRecovery2` is traced by HDFS-16853.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1368240999
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 55s | | trunk passed |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 31s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 17s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 7s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 383m 1s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 496m 32s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestLeaseRecovery2 |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5264 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e4c1bbe0ebc5 4.15.0-197-generic #208-Ubuntu SMP Tue Nov 1 17:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 18a55dd577f32e2010de118c13533e9150eb1302 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/testReport/ |
| Max. process+thread count | 2217 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ZanderXu commented on pull request #5264: HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks
Posted by GitBox <gi...@apache.org>.
ZanderXu commented on PR #5264:
URL: https://github.com/apache/hadoop/pull/5264#issuecomment-1370453598
Merged, thanks @haiyang1987 for your report. Thanks @dineshchitlangia for your review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org