You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/01/31 22:47:36 UTC
[GitHub] [hbase] apurtell opened a new pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
apurtell opened a new pull request #4087:
URL: https://github.com/apache/hbase/pull/4087
We have encountered two issues with region warmup before the assignment manager gracefully moves a region from one regionserver to another.
The first instance is HBASE-26722. Part of the failure chain is temporary double assignment like conditions where both the source and destination regionservers think they have exclusive rights to storefile management (as they should) but warmup opens a region before it is closed and then both regionservers take compaction related actions. While this can be remediated with more care to this case, it is unclear if warmup affords significant advantage. The motivation of the original commit in 2015 introducing this feature was avoidance of blockcache misses once region ownership transfer was advertised to clients. Depending on use case and additional default-false schema options (like preload) this could be valuable. Or not.
The second instance is in place upgrade from HBase 1 to HBase 2. In a scenario where regionservers have been replaced by HBase 2 versions, but the master is still HBase 1, the region warmup RPC request fails. This is not particularly harmful but indicates it will not be useful during the transition period.
We think it would be good to allow for warmup before move to be optionally disabled by a site configuration setting. In particular there have been many unrelated changes committed since 2015 and expectations of invariants in the contribution of the warmup-on-move feature have been invalidated. HBASE-26722 may be the only case, or it might not. This is a fairly trivial change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #4087:
URL: https://github.com/apache/hbase/pull/4087#issuecomment-1027143194
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 46s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-1 Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 35s | branch-1 passed |
| +1 :green_heart: | compile | 0m 45s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 0m 49s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 1m 44s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 3m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 3s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 12s | the patch passed |
| +1 :green_heart: | compile | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 0m 44s | the patch passed |
| +1 :green_heart: | compile | 0m 47s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 0m 47s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 32s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 3m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 5m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 2m 59s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 133m 47s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. |
| | | 192m 38s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.replication.TestReplicationSource |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4087 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b42564ddc209 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4087/out/precommit/personality/provided.sh |
| git revision | branch-1 / a2406f3764 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/2/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/2/testReport/ |
| Max. process+thread count | 4403 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #4087:
URL: https://github.com/apache/hbase/pull/4087#issuecomment-1027394622
Thanks.
Agreed, the test failures are not relevant. The default preserves existing behavior, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #4087:
URL: https://github.com/apache/hbase/pull/4087#issuecomment-1026769128
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 10m 12s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ branch-1 Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 31s | branch-1 passed |
| +1 :green_heart: | compile | 0m 51s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | compile | 0m 55s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | checkstyle | 2m 2s | branch-1 passed |
| +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 44s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +0 :ok: | spotbugs | 3m 31s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 28s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 31s | the patch passed |
| +1 :green_heart: | compile | 0m 51s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javac | 0m 51s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 46s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 3m 26s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 5m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 33s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 |
| +1 :green_heart: | findbugs | 3m 22s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 176m 13s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. |
| | | 234m 0s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
| | hadoop.hbase.master.TestMasterOperationsForRegionReplicas |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4087 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 846572b84453 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-4087/out/precommit/personality/provided.sh |
| git revision | branch-1 / a2406f3764 |
| Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/1/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/1/testReport/ |
| Max. process+thread count | 4939 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4087/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
Posted by GitBox <gi...@apache.org>.
apurtell merged pull request #4087:
URL: https://github.com/apache/hbase/pull/4087
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #4087: HBASE-26726 Allow disable of region warmup before graceful move
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #4087:
URL: https://github.com/apache/hbase/pull/4087#issuecomment-1026919559
Re-triggered the QA as test failures should not be ideally relevant to this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org