You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/05/30 14:47:08 UTC
[GitHub] [hbase] busbey opened a new pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
busbey opened a new pull request #1813:
URL: https://github.com/apache/hbase/pull/1813
These changes assume that we're running a single release build at at time and that primarily we only need to see stuff outside of the container when it is done.
docs on delegated consistency:
https://docs.docker.com/storage/bind-mounts/#configure-mount-consistency-for-macos
Before this change (using my branch with HBASE-23339 in place to build an RC from master):
```
busbey$ grep -E "(Total time:|uilding binary dist)" output/publish-dist.log
+ echo '2020-05-16T07:35:10Z Building binary dist'
2020-05-16T07:35:10Z Building binary dist
08:46:50 [INFO] Total time: 01:11 h
11:58:02 [INFO] Total time: 03:11 h
16:33:11 [INFO] Total time: 04:34 h
+ echo '2020-05-16T16:35:40Z Done building binary distribution'
2020-05-16T16:35:40Z Done building binary distribution
```
and then the final deploy took a bit over 5 hours.
after this change we see a pretty dramatic improvement:
```
busbey$ grep -E "(Total time:|uilding binary dist)" output/publish-dist.log
+ echo '2020-05-29T22:54:08Z Building binary dist'
2020-05-29T22:54:08Z Building binary dist
23:17:49 [INFO] Total time: 23:34 min
23:37:57 [INFO] Total time: 19:56 min
00:49:06 [INFO] Total time: 01:10 h
+ echo '2020-05-30T00:51:52Z Done building binary distribution'
2020-05-30T00:51:52Z Done building binary distribution
busbey$ grep -E "(Total time:|uilding binary dist)" output/mvn_deploy_release.log
03:08:57 [INFO] Total time: 47:13 min
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1813:
URL: https://github.com/apache/hbase/pull/1813#issuecomment-636341896
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch |
||| _ Other Tests _ |
| | | 1m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.10 Server=19.03.10 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1813 |
| Optional Tests | |
| uname | Linux 38e4623f891a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 36a6ef9cf9 |
| Max. process+thread count | 52 (vs. ulimit of 12500) |
| modules | C: U: |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] busbey commented on pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
busbey commented on pull request #1813:
URL: https://github.com/apache/hbase/pull/1813#issuecomment-639190059
thanks for the reviews y'all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] asfgit closed pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1813:
URL: https://github.com/apache/hbase/pull/1813
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1813:
URL: https://github.com/apache/hbase/pull/1813#issuecomment-636341930
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch |
| +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
||| _ Other Tests _ |
| +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? |
| | | 1m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.10 Server=19.03.10 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1813 |
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 62efc9c1651b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 36a6ef9cf9 |
| Max. process+thread count | 48 (vs. ulimit of 12500) |
| modules | C: U: |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1813:
URL: https://github.com/apache/hbase/pull/1813#issuecomment-636342016
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 23s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch |
||| _ Other Tests _ |
| | | 2m 30s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.10 Server=19.03.10 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1813 |
| Optional Tests | |
| uname | Linux 08f3dc2c0f63 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 36a6ef9cf9 |
| Max. process+thread count | 55 (vs. ulimit of 12500) |
| modules | C: U: |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1813/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] busbey commented on pull request #1813: HBASE-24458 release scripts using docker should specify needed disk consistency
Posted by GitBox <gi...@apache.org>.
busbey commented on pull request #1813:
URL: https://github.com/apache/hbase/pull/1813#issuecomment-636413619
> Perhaps add a quick comment that it is an osx optimization and ignored with other OSes?
I didn't do that initially because that's an implementation detail of docker. the consistency statement could be used by docker on other platforms in the future. let me try including something about the OS X optimization on why we're doing it specifically.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org