You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "luoyuan3471 (via GitHub)" <gi...@apache.org> on 2026/09/29 12:45:35 UTC
[PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
luoyuan3471 opened a new pull request, #8767:
URL: https://github.com/apache/hadoop/pull/8767
### Description of PR
When multiple containers request the same Docker image concurrently, separate NodeManager launch threads can invoke duplicate `docker pull` commands for that image. These pulls contend on the same image and perform redundant work.
This change serializes pulls by image name with reference-counted per-image locks. Pulls for the same image execute one at a time, while pulls for different images can still run concurrently. Unused locks are removed after the last waiter finishes, including when a pull fails.
JIRA: https://issues.apache.org/jira/browse/YARN-11997
### How was this patch tested?
- Added focused coverage in `TestDockerContainerRuntime` for serializing concurrent pulls of the same image, allowing different images to pull concurrently, and releasing the image lock after a failed pull.
- `TestDockerContainerRuntime`: 127 tests passed, 0 failures, 0 errors, 0 skipped.
- `git diff --check` passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "luoyuan3471 (via GitHub)" <gi...@apache.org>.
luoyuan3471 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6282146094
Hi @slfan1989, friendly ping — all checks are green now. Would appreciate your review when convenient. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6120858296
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 46s | | trunk passed |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 2m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 30m 37s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 29m 40s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 26m 32s | | hadoop-yarn-server-nodemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not generate ASF License warnings. |
| | | 152m 17s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8767 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c45779266888 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fe8111e461dc8c6fdf7befdfc0f6dc7f537b2741 |
| Default Java | Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/3/testReport/ |
| Max. process+thread count | 611 (vs. ulimit of 10000) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/3/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "luoyuan3471 (via GitHub)" <gi...@apache.org>.
luoyuan3471 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-5908341048
Hi @brumi1024, could you please take a look when you have time? This change serializes concurrent Docker image pulls by image name while still allowing different images to be pulled in parallel, preventing duplicate pulls and related races in the NodeManager. It includes focused concurrency tests. The patch-side compile, SpotBugs, and unit tests passed; the current overall -1 is caused by an existing SpotBugs warning on trunk. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "luoyuan3471 (via GitHub) via common-issues" <co...@hadoop.apache.org>.
luoyuan3471 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6652182206
Hi @pan3793, could you also review this NodeManager Docker image-pull concurrency fix when convenient? It includes regression tests, and all CI checks are green. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "luoyuan3471 (via GitHub)" <gi...@apache.org>.
luoyuan3471 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6136974410
@slfan1989 could you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6434749840
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 13m 13s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 40s | | trunk passed |
| +1 :green_heart: | compile | 1m 35s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 34s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 34m 8s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 38s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 30m 22s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 26m 51s | | hadoop-yarn-server-nodemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 168m 37s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/5/artifact/out/Dockerfile |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fd03dfc985fe 5.15.0-198-generic #208-Ubuntu SMP Fri Sep 4 10:22:38 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f842297570243587f0ed7a81ebe80bb178098bca |
| Default Java | Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/5/testReport/ |
| Max. process+thread count | 635 (vs. ulimit of 10000) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/5/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "luoyuan3471 (via GitHub)" <gi...@apache.org>.
luoyuan3471 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6516861748
Hi @slfan1989, gentle follow-up on this PR. The latest revision has green Yetus and CI checks. Could you take a look when you have time? Happy to address any feedback. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6120904395
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 41s | | trunk passed |
| +1 :green_heart: | compile | 1m 39s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 30m 14s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 30m 48s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 36s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 29m 21s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 26m 30s | | hadoop-yarn-server-nodemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 151m 55s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/4/artifact/out/Dockerfile |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c077f60e359b 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3aeb542f5df374444ce47caa3a8f73eb3743efe6 |
| Default Java | Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/4/testReport/ |
| Max. process+thread count | 635 (vs. ulimit of 10000) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/4/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-5893109029
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 34s | | trunk passed |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 11s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| -1 :x: | spotbugs | 1m 57s | [/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/1/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 29m 30s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 30m 1s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 7s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 37s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 35s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 26m 18s | | hadoop-yarn-server-nodemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. |
| | | 148m 24s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/1/artifact/out/Dockerfile |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d45ea02a9e1e 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8545d5e3a949746f5f23a9587d5cad8e72d2e741 |
| Default Java | Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/1/testReport/ |
| Max. process+thread count | 631 (vs. ulimit of 10000) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/1/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6118059983
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 46s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 10s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 13s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 29m 42s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 1m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 26m 37s | | hadoop-yarn-server-nodemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not generate ASF License warnings. |
| | | 150m 33s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8767 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5940669b9a4e 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 600af1657335d391660ccefcb7cebe714b197ba6 |
| Default Java | Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/2/testReport/ |
| Max. process+thread count | 639 (vs. ulimit of 10000) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8767/2/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11997. Serialize concurrent Docker image pulls by image name [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #8767:
URL: https://github.com/apache/hadoop/pull/8767#issuecomment-6137260198
> @slfan1989 could you take a look?
Thank you for your contribution. I'm a bit tied up today, but I'll review this PR within the next day or two.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org