You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "GauthamBanasandra (via GitHub)" <gi...@apache.org> on 2023/05/21 07:58:28 UTC
[GitHub] [hadoop] GauthamBanasandra opened a new pull request, #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
GauthamBanasandra opened a new pull request, #5679:
URL: https://github.com/apache/hadoop/pull/5679
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Currently, mvnsite build phase fails due to the following error -
```
[INFO] ------------------< org.apache.hadoop:hadoop-common >-------------------
[INFO] Building Apache Hadoop Common 3.4.0-SNAPSHOT [11/114]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ hadoop-common ---
[INFO] Deleting C:\hadoop\hadoop-common-project\hadoop-common\target
[INFO] Deleting C:\hadoop\hadoop-common-project\hadoop-common\src\site\markdown (includes = [UnixShellAPI.md], excludes = [])
[INFO] Deleting C:\hadoop\hadoop-common-project\hadoop-common\src\site\resources (includes = [configuration.xsl, core-default.xml], excludes = [])
[INFO]
[INFO] --- exec-maven-plugin:1.3.1:exec (shelldocs) @ hadoop-common ---
tar: apache-yetus-0.14.0/lib/precommit/qbt.sh: Cannot create symlink to 'test-patch.sh': No such file or directory
tar: Exiting with failure status due to previous errors
/usr/bin/env: 'python3': No such file or directory
```
This PR installs Python 3 in the Windows 10 Hadoop builder docker image to fix this.
### How was this patch tested?
Jenkins CI. Build results -
1. Without this PR - [patch-mvnsite-root-fail.txt](https://github.com/apache/hadoop/files/11523971/patch-mvnsite-root-fail.txt)
2. With this PR - [patch-mvnsite-root-pass.txt](https://github.com/apache/hadoop/files/11523970/patch-mvnsite-root-pass.txt)
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a w
[patch-mvnsite-root-fail.txt](https://github.com/apache/hadoop/files/11523969/patch-mvnsite-root-fail.txt)
ay that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #5679:
URL: https://github.com/apache/hadoop/pull/5679#discussion_r1199764494
##########
dev-support/docker/Dockerfile_windows_10:
##########
@@ -108,6 +108,12 @@ RUN powershell Copy-Item -Path "C:\LibXXHash\usr\bin\*.dll" -Destination "C:\Pro
RUN powershell Copy-Item -Path "C:\LibZStd\usr\bin\*.dll" -Destination "C:\Program` Files\Git\usr\bin"
RUN powershell Copy-Item -Path "C:\RSync\usr\bin\*" -Destination "C:\Program` Files\Git\usr\bin"
+# Install Python 3.10.11.
Review Comment:
nopes, just wanted to know, if we have any challenges with the higher versions. Good with me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "GauthamBanasandra (via GitHub)" <gi...@apache.org>.
GauthamBanasandra commented on code in PR #5679:
URL: https://github.com/apache/hadoop/pull/5679#discussion_r1199758698
##########
dev-support/docker/Dockerfile_windows_10:
##########
@@ -108,6 +108,12 @@ RUN powershell Copy-Item -Path "C:\LibXXHash\usr\bin\*.dll" -Destination "C:\Pro
RUN powershell Copy-Item -Path "C:\LibZStd\usr\bin\*.dll" -Destination "C:\Program` Files\Git\usr\bin"
RUN powershell Copy-Item -Path "C:\RSync\usr\bin\*" -Destination "C:\Program` Files\Git\usr\bin"
+# Install Python 3.10.11.
Review Comment:
I normally choose a version that's less recent so that it's mature and has got the appropriate fixes (if any that were discovered upon release).
1. `3.10.11` -> Would've gone through 11 iterations after release, hopefully containing the bug fixes from the time when `3.10.0` was released.
2. `3.12.0` -> Is just the first release and could potentially have some bugs.
This is just a practice that I follow. I don't mind bumping up the version to `3.12.0` if you insist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5679:
URL: https://github.com/apache/hadoop/pull/5679#issuecomment-1556128063
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 16m 56s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 38m 23s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | shadedclient | 19m 20s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 78m 27s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5679/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5679 |
| Optional Tests | dupname asflicense |
| uname | Linux f698310145f3 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 631c73959fc311d01665de491c461b48fdbb619a |
| Max. process+thread count | 557 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5679/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra merged pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "GauthamBanasandra (via GitHub)" <gi...@apache.org>.
GauthamBanasandra merged PR #5679:
URL: https://github.com/apache/hadoop/pull/5679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #5679:
URL: https://github.com/apache/hadoop/pull/5679#discussion_r1199750794
##########
dev-support/docker/Dockerfile_windows_10:
##########
@@ -108,6 +108,12 @@ RUN powershell Copy-Item -Path "C:\LibXXHash\usr\bin\*.dll" -Destination "C:\Pro
RUN powershell Copy-Item -Path "C:\LibZStd\usr\bin\*.dll" -Destination "C:\Program` Files\Git\usr\bin"
RUN powershell Copy-Item -Path "C:\RSync\usr\bin\*" -Destination "C:\Program` Files\Git\usr\bin"
+# Install Python 3.10.11.
Review Comment:
curious any specific reason for using 3.10.11? If I see here:
https://www.python.org/ftp/python/
we have 3.12.0 as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5679: HADOOP-18746. Install Python 3 for Windows 10 docker image
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5679:
URL: https://github.com/apache/hadoop/pull/5679#issuecomment-1556112446
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5679/1/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org