You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "arhxam (via GitHub)" <gi...@apache.org> on 2026/08/05 23:24:44 UTC
[PR] HADOOP-19387. Recover LocalDirAllocator after failed initialization [hadoop]
arhxam opened a new pull request, #8660:
URL: https://github.com/apache/hadoop/pull/8660
### Description of PR
Fixes [HADOOP-19387](https://issues.apache.org/jira/browse/HADOOP-19387).
`LocalDirAllocator` currently records a directory configuration as initialized even when every configured directory fails initialization. An interruption can therefore leave an empty directory context cached indefinitely: later calls see the unchanged configuration and never retry.
This patch records `savedLocalDirs` only after at least one directory is successfully initialized. A subsequent call can then retry a transiently failed initialization. The regression test uses a one-shot `DiskValidator` failure to exercise the same empty-context path deterministically, without relying on process timing.
This supersedes the stale, closed attempt in #7298 and updates the regression coverage to the current JUnit 5 test suite.
### How was this patch tested?
On macOS with OpenJDK 17.0.20 and Maven Wrapper 3.9.15:
```text
./mvnw -pl hadoop-common-project/hadoop-common -am -DskipShade \
-Dtest=TestLocalDirAllocator \
-Dsurefire.failIfNoSpecifiedTests=false test
Tests run: 49, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
```
Before the production change, the new test fails on the second allocation with `DiskErrorException`, confirming the regression.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id?
- [ ] Object storage: N/A
- [ ] If adding new dependencies: no new dependencies
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? N/A
### AI Tooling
Contains content generated by Codex.
- [x] The PR includes the phrase "Contains content generated by Codex"
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19387. Recover LocalDirAllocator after failed initialization [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8660:
URL: https://github.com/apache/hadoop/pull/8660#issuecomment-5199388322
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 26m 37s | | trunk passed |
| +1 :green_heart: | compile | 8m 55s | | trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | compile | 8m 51s | | trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 11s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 8s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 8m 23s | | the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | javac | 8m 23s | | the patch passed |
| +1 :green_heart: | compile | 8m 49s | | the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | javac | 8m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 1s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 40s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not generate ASF License warnings. |
| | | 128m 59s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8660/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8660 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8d3b6edb8db0 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b9158dca520f3826d905d87941ca68e80ec337ed |
| Default Java | Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8660/1/testReport/ |
| Max. process+thread count | 1313 (vs. ulimit of 10000) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8660/1/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org