You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/05/21 09:18:33 UTC
[GitHub] [hadoop] jojochuang opened a new pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
jojochuang opened a new pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037
HADOOP-17723
verified manually by running
`dev-support/bin/create-release --asfrelease --docker --dockercache` on an ARM machine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846040629
> > This is broken by #1738
>
> That was awhile ago. Gap in our CI coverage for ARM?
I guess the ARM nightly build doesn't use the docker image? Or it has been failing for a while? Because apparently the python3 migration wasn't successful. Other than this, yarn-ui profile still requires python2. Additionally, part of the code requires phantomJS where the ARM binary was built for Ubuntu Xenial, not Bionic (missing dynamic libs only available in Xenial.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] busbey commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
busbey commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846229893
Ah. Yeah looks like despite being named with the pattern we use for Yetus driven CI, the arm job is just a bash script wrapping a maven call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-845839714
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 13m 18s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | hadolint | 0m 1s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 12m 54s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not generate ASF License warnings. |
| | | 45m 43s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3037 |
| Optional Tests | dupname asflicense codespell hadolint shellcheck shelldocs |
| uname | Linux 42e29ca53150 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dd01ced464bb85bfe9fef5657e2872a1e5618474 |
| Max. process+thread count | 680 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/1/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 hadolint=1.11.1-0-g0e692dd |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-845812463
@aajisaka @brahmareddybattula @vinayakumarb could you review the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#discussion_r637768966
##########
File path: dev-support/docker/Dockerfile_aarch64
##########
@@ -80,12 +80,14 @@ RUN apt-get -q update \
python3-pkg-resources \
python3-setuptools \
python3-wheel \
+ python2.7 \
rsync \
shellcheck \
software-properties-common \
sudo \
valgrind \
zlib1g-dev \
+ phantomjs \
Review comment:
The package list should be sorted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846906952
> Ah. Yeah looks like despite being named with the pattern we use for Yetus driven CI, the arm job is just a bash script wrapping a maven call.
The reason for this was, at that time, yetus was not supporting aarch64 CIs. Not sure whether latest Yetus supports aarch64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#discussion_r637344440
##########
File path: dev-support/docker/Dockerfile_aarch64
##########
@@ -169,6 +160,9 @@ ENV MAVEN_OPTS -Xms256m -Xmx1536m
# Skip gpg verification when downloading Yetus via yetus-wrapper
ENV HADOOP_SKIP_YETUS_VERIFICATION true
+# Force PhantomJS to be in 'headless' mode, do not connect to Xwindow
+ENV QT_QPA_PLATFORM offscreen
+
Review comment:
The official Ubuntu phantomjs has a caveat: it requires X Window. (ariya/phantomjs#14376 (comment)). Add environment var QT_QPA_PLATFORM=offscreen to make it really 'headless". If it's not headless, the installer fails to execute phantomjs and abort.
##########
File path: dev-support/docker/Dockerfile_aarch64
##########
@@ -150,17 +152,6 @@ RUN pip3 install pylint==2.6.0 python-dateutil==2.8.1
# hadolint ignore=DL3008
RUN npm install -g bower@1.8.8
-###
Review comment:
replaced the custom PhantomJS binary (which was built for Xenial and does not run on Bionic) with the official phantomjs package for Bionic.
##########
File path: dev-support/docker/Dockerfile_aarch64
##########
@@ -80,12 +80,14 @@ RUN apt-get -q update \
python3-pkg-resources \
python3-setuptools \
python3-wheel \
+ python2.7 \
Review comment:
We need python2.7 package, because hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui installs node-gyp 3.8, which has dependency on python2 to build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846943173
Merged. Will cherrypick to branch-3.3.1. Thanks @aajisaka, @busbey and @vinayakumarb for review and inputs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846875277
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/3/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846970564
Thanks, @jojochuang. I got no issue on building and testing of yarn-ui on EC2 A1 instance with the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846038595
There exists a daily ARM qbt job, but it does not use the docker image: https://ci-hadoop.apache.org/job/hadoop-qbt-linux-ARM-trunk/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846343152
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 8m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 13m 8s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | hadolint | 0m 3s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 12m 48s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not generate ASF License warnings. |
| | | 53m 53s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3037 |
| Optional Tests | dupname asflicense codespell hadolint shellcheck shelldocs |
| uname | Linux 6d5abe4bbfd0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6354405d3ff9372ba351240046b5e687c9b1e6aa |
| Max. process+thread count | 720 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/2/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 hadolint=1.11.1-0-g0e692dd |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-845813710
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/1/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846902985
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 7s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 13m 4s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 12m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not generate ASF License warnings. |
| | | 45m 29s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3037 |
| Optional Tests | dupname asflicense codespell hadolint shellcheck shelldocs |
| uname | Linux b5eb6dd19d2f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 35ea183490deca47531ee19cb74de760410117a7 |
| Max. process+thread count | 658 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/3/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 hadolint=1.11.1-0-g0e692dd |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-845824398
This is broken by #1738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-845828349
WIP. Please wait for me to run through the whole release creation process for ARM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846338046
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3037/2/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3037: HADOOP-17723. [build] python3 migration broke Docker container image for ARM.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037#issuecomment-846339358
Installs python2.7, and replaced the custom PhantomJS binary (which was built for Xenial and does not run on Bionic) with the official phantomjs package for Bionic.
Note: Yes I am aware we migrated to Python3. However, the installation of nodejs implicitly installs python-minimal, which is python2 without lots of packages. So python2 is installed by the ARM Docker image anyway (possibly the x86 docker image has it too).
The official Ubuntu phantomjs has a caveat: it requires X Window. (https://github.com/ariya/phantomjs/issues/14376#issuecomment-239687524). Add environment var QT_QPA_PLATFORM=offscreen to make it really 'headless". If it's not headless, the installer fails to execute phantomjs and abort.
In comparision, the x86 Dockerfile doesn't install the official Ubuntu phantomjs package. Instead, it installs the version managed by the npm/yarn package manager.
We need python2.7 package, because hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui installs node-gyp 3.8, which has dependency on python2 to build. Not clear to me why the x86 Dockerfile doesn't require it. My guess is that the node-gyp has official x86 binary so it doesn't need to be built from source.
That's a lot of digest in this comment since it involves Docker, Ubuntu, Maven, yarn, Python. Lots of different package managers. But at the end of it I was able to build for ARM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang merged pull request #3037: HADOOP-17723. [build] fix the Dockerfile for ARM
Posted by GitBox <gi...@apache.org>.
jojochuang merged pull request #3037:
URL: https://github.com/apache/hadoop/pull/3037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org