You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/02/01 15:47:20 UTC
[GitHub] [hadoop] GauthamBanasandra opened a new pull request #2669: HADOOP-17509. Parallilize building of deps
GauthamBanasandra opened a new pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669
* Added -j option to parallelize the
building of Protocol buffers and Intel
ISA-L dependencies.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771451648
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773599937
@GauthamBanasandra Yes. I just ran docker build again and it still fails, seemingly because of boost library compilation failure:
```
common.copy /usr/lib/libboost_wave.a
...failed updating 11 targets...
...skipped 24 targets...
...updated 16944 targets...
The command '/bin/bash -o pipefail -c mkdir -p /opt/boost-library && curl -L https://sourceforge.net/projects/boost/files/boost/1.72.0/boost_1_72_0.tar.bz2/download > boost_1_72_0.tar.bz2 && mv boost_1_72_0.tar.bz2 /opt/boost-library && cd /opt/boost-library && tar --bzip2 -xf boost_1_72_0.tar.bz2 && cd /opt/boost-library/boost_1_72_0 && ./bootstrap.sh --prefix=/usr/ && ./b2 --without-python install && cd /root && rm -rf /opt/boost-library' returned a non-zero code: 1
```
This should have nothing to do with this PR. Don't worry. :)
I posted the full log [here](https://gist.github.com/smengcl/200668f706306031f72b8010d7c728a0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771702192
Thank you @GauthamBanasandra for your contribution!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra edited a comment on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
GauthamBanasandra edited a comment on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773073367
Hi @smengcl ,
Are you saying that you encountered an error while doing a docker build on macOS? If so, could you please post the error here?
I don't own a mac. I'll try to see if I can spin one up on the cloud and verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771478842
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 4m 48s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 16s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 13m 27s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | hadolint | 0m 4s | | There were no new hadolint issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 20s | | There were no new shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 12m 56s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not generate ASF License warnings. |
| | | 48m 57s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2669 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs |
| uname | Linux abf3a8386921 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9bf2ac07bb8 |
| Max. process+thread count | 677 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/2/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 hadolint=1.11.1-0-g0e692dd |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
aajisaka merged pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-781981233
@GauthamBanasandra Good find!
I wonder if the reason is Mac Docker's mem limit.
-- Yes. The build succeeded after I increased the Docker global memory limit from 3GB to 5GB (4GB doesn't work). Wonder why I haven't hit the mem limit before 🤔
Thanks for the help :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771702192
Thank you @GauthamBanasandra for your contribution!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
GauthamBanasandra commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773073367
Hi @smengcl ,
Are you saying that you encountered an error while doing a docker build on macOS? If so, could you please post the error here?
I don't own a mac. I'll try to see if I spin one up on the cloud and verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2669: HADOOP-17509. Parallilize building of deps
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-770955554
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/1/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra edited a comment on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
GauthamBanasandra edited a comment on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773073367
Hi @smengcl ,
Are you saying that you encountered an error while doing a docker build on macOS? If so, could you please post the error here?
I don't own a mac. I'll try to see if I can spin one up on the cloud and verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771451648
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/2/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
GauthamBanasandra commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773073367
Hi @smengcl ,
Are you saying that you encountered an error while doing a docker build on macOS? If so, could you please post the error here?
I don't own a mac. I'll try to see if I spin one up on the cloud and verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-772910476
Hey @GauthamBanasandra sorry I'm late to this.
I can verify that on Ubuntu 20.04 I am able to `docker build .` with the change. But **not** on macOS 10.15.7 (should be fine for now).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2669: HADOOP-17509. Parallilize building of deps
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-771004420
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 10m 18s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 23m 31s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| -1 :x: | hadolint | 0m 4s | [/diff-patch-hadolint.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/1/artifact/out/diff-patch-hadolint.txt) | The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 15s | | There were no new shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 20m 6s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not generate ASF License warnings. |
| | | 72m 22s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2669 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs |
| uname | Linux e334271442f7 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b54134661b5 |
| Max. process+thread count | 572 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2669/1/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 hadolint=1.11.1-0-g0e692dd |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
GauthamBanasandra commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-776506622
@smengcl I just had a look at the logs. I couldn't find the exact error but I found this - https://gist.github.com/smengcl/200668f706306031f72b8010d7c728a0#file-docker_build-log-L777-L778
I'm not sure if this is a kill signal that was sent externally. Would you happen to know the exact reason for compilation failure?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2669: HADOOP-17509. Parallelize building of deps
Posted by GitBox <gi...@apache.org>.
aajisaka merged pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on pull request #2669: HADOOP-17509. Parallelize building of dependencies
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #2669:
URL: https://github.com/apache/hadoop/pull/2669#issuecomment-773599937
@GauthamBanasandra Yes. I just ran docker build again and it still fails, seemingly because of boost library compilation failure:
```
common.copy /usr/lib/libboost_wave.a
...failed updating 11 targets...
...skipped 24 targets...
...updated 16944 targets...
The command '/bin/bash -o pipefail -c mkdir -p /opt/boost-library && curl -L https://sourceforge.net/projects/boost/files/boost/1.72.0/boost_1_72_0.tar.bz2/download > boost_1_72_0.tar.bz2 && mv boost_1_72_0.tar.bz2 /opt/boost-library && cd /opt/boost-library && tar --bzip2 -xf boost_1_72_0.tar.bz2 && cd /opt/boost-library/boost_1_72_0 && ./bootstrap.sh --prefix=/usr/ && ./b2 --without-python install && cd /root && rm -rf /opt/boost-library' returned a non-zero code: 1
```
This should have nothing to do with this PR. Don't worry. :)
I posted the full log [here](https://gist.github.com/smengcl/200668f706306031f72b8010d7c728a0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org