You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "stoty (via GitHub)" <gi...@apache.org> on 2025/03/12 08:45:07 UTC
[PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
stoty opened a new pull request, #7499:
URL: https://github.com/apache/hadoop/pull/7499
also fix Centos 7 docker image build
### Description of PR
Revert io.grpc:protoc-gen-grpc-java to 1.65.0 to enable building on Centos 7.
Also backported Centos 7 image build fix from trunk.
### How was this patch tested?
Started Centos7 container.
ran 'mvn clean package -DskipTests'
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991825517
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
> There are no code changes in the compiler between those versions
it's fine for these specific version pair, but may not be good for future upgrading
> committing generated code to git is troublesome, I would very much like to avoid that.
I tend to agree with you in most cases, but I think for this case, it's a most safe and relatively cheap way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "stoty (via GitHub)" <gi...@apache.org>.
stoty commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991843774
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
> it's fine for these specific version pair, but may not be good for future upgrading
>
We can cross that rover when we get there.
If you REALLY don't want to use different versions, then I'd preder recompiling the plugin from source on Centos 7, though I'd hate to put in that much effort to support EOL distros.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#issuecomment-2717310650
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 7m 6s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ branch-3.4 Compile Tests _ |
| +1 :green_heart: | mvninstall | 22m 11s | | branch-3.4 passed |
| +1 :green_heart: | compile | 0m 22s | | branch-3.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 20s | | branch-3.4 passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | mvnsite | 0m 22s | | branch-3.4 passed |
| +1 :green_heart: | javadoc | 0m 24s | | branch-3.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 20s | | branch-3.4 passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | shadedclient | 19m 19s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 0m 15s | | the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 15s | | the patch passed |
| +1 :green_heart: | compile | 0m 14s | | the patch passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 17s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | shadedclient | 19m 8s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | | hadoop-yarn-csi in the patch passed. |
| +1 :green_heart: | asflicense | 0m 25s | | The patch does not generate ASF License warnings. |
| | | 72m 24s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7499 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint |
| uname | Linux e364faa10d57 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.4 / 8a80d59349406a07da67c7a730f87e1b5a26b649 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/testReport/ |
| Max. process+thread count | 552 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7499/1/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1992464311
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
Thanks to both of you for the discussion! In the current situation, I lean towards supporting Story's approach, as long as it doesn't block the progress of Hadoop 3.4.2. In fact, we don't actually need this CentOS7 script during the release process. If a similar issue arises in a future version of Ubuntu, I believe we can follow @pan3793 method to fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991495956
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
It looks risky to use a different gRPC version for codegen and runtime, an alternative I can imagine -track the generated source code in Git, and regenerate the code on each gRPC upgrading
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#issuecomment-2731769768
I am not against the current approach as a workaround for branch-3.4, suppose there will be no upgrading of gRPC on branch-3.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "stoty (via GitHub)" <gi...@apache.org>.
stoty commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991853859
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
Yes, that's what I meant by recompiling the grpc compiler from source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #7499: YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image
URL: https://github.com/apache/hadoop/pull/7499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#issuecomment-2732508604
I'd prefer committing generated code to source; much safer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991861477
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
BTW, Hadoop actually has committed a few generated code into Git repo
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/arm-java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#issuecomment-3283055677
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991495956
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
It looks risky to use a different gRPC version for codegen and runtime, an alternative I can imagine -track the generated source code in Git, and only regenerate the code on each gRPC upgrading
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991850497
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
Another way used by Spark, allow user to configure pre-installed `protoc`/`grpc` commands to generate the code, then we can compile and install those commands from the source code on the CentOS 7 build image (I'm not familiar with the native world, so have no idea how much maintenance effort will be paid for this way)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] YARN-11794. hadoop-yarn-csi does not build on included Centos 7 docker image [hadoop]
Posted by "stoty (via GitHub)" <gi...@apache.org>.
stoty commented on code in PR #7499:
URL: https://github.com/apache/hadoop/pull/7499#discussion_r1991720404
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml:
##########
@@ -27,6 +27,8 @@
<properties>
<grpc.version>1.69.0</grpc.version>
+ <!-- Last version that works with Centos 7 -->
+ <grpc.plugin.version>1.65.0</grpc.plugin.version>
Review Comment:
There are no code changes in the compiler between those versions , @pan3793
Only the build OS is changed, event the netty version change is reverted:
https://github.com/grpc/grpc-java/commits/master/compiler
The incompatibility is caused by
https://github.com/grpc/grpc-java/commit/71eb5fb9f0e408e8c7e213ef470a8894ee0a4e18
committing generated code to git is troublesome, I would very much like to avoid that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org