You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/03/11 14:42:20 UTC
[GitHub] [hbase] mwkang opened a new pull request #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
mwkang opened a new pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274
The compaction check that the system is stopped when it reaches hbase.hstore.close.check.interval (size) or hbase.hstore.close.check.time.interval (time)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605848606
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 11s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | shadedjars | 5m 23s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 47s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 21s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 156m 39s | hbase-server in the patch failed. |
| | | 186m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 3c72ba2be659 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d4e020c3d |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/testReport/ |
| Max. process+thread count | 3862 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605744215
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396669567
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
Review comment:
Does it have to be public? Can it be package local? It is only used in this package?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605784595
@busbey Thank you!!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610151811
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 13s | master passed |
| +1 :green_heart: | compile | 1m 2s | master passed |
| -1 :x: | shadedjars | 0m 15s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 54s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| -1 :x: | shadedjars | 0m 12s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 114m 38s | hbase-server in the patch passed. |
| | | 133m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 33181e872a69 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 15b94cc306 |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/testReport/ |
| Max. process+thread count | 3644 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597769073
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 40s | master passed |
| +1 :green_heart: | compile | 0m 57s | master passed |
| +1 :green_heart: | shadedjars | 4m 48s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 68m 44s | hbase-server in the patch passed. |
| | | 94m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 09be94789ba5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/testReport/ |
| Max. process+thread count | 8851 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606302097
Hard to tell. It failed making the report. See below. Let me try locally and if good I'll commit.
```
[2020-03-30T08:09:56.684Z] ============================================================================
[2020-03-30T08:09:56.684Z] ============================================================================
[2020-03-30T08:09:56.684Z] Finished build.
[2020-03-30T08:09:56.684Z] ============================================================================
[2020-03-30T08:09:56.684Z] ============================================================================
[2020-03-30T08:09:56.684Z]
[2020-03-30T08:09:56.684Z]
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // dir
Post stage
[Pipeline] junit
[2020-03-30T08:09:57.280Z] Recording test results
[2020-03-30T08:09:59.179Z] Remote call on H1 failed
Error when executing always post condition:
java.io.IOException: Remote call on H1 failed
at hudson.remoting.Channel.call(Channel.java:963)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:114)
at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:137)
at hudson.tasks.junit.JUnitResultArchiver.parseAndAttach(JUnitResultArchiver.java:167)
at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:52)
at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:25)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins
at hudson.ExtensionList.lookup(ExtensionList.java:433)
at hudson.tasks.junit.TestNameTransformer.all(TestNameTransformer.java:40)
at hudson.tasks.junit.TestNameTransformer.getTransformedName(TestNameTransformer.java:33)
at hudson.tasks.junit.CaseResult.getTransformedTestName(CaseResult.java:273)
at hudson.tasks.junit.SuiteResult.casesByName(SuiteResult.java:134)
at hudson.tasks.junit.SuiteResult.addCase(SuiteResult.java:297)
at hudson.tasks.junit.SuiteResult.<init>(SuiteResult.java:270)
at hudson.tasks.junit.SuiteResult.parseSuite(SuiteResult.java:209)
at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:181)
at hudson.tasks.junit.TestResult.parse(TestResult.java:348)
at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:281)
at hudson.tasks.junit.TestResult.parse(TestResult.java:206)
at hudson.tasks.junit.TestResult.parse(TestResult.java:178)
at hudson.tasks.junit.TestResult.<init>(TestResult.java:143)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:146)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:118)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
... 4 more
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to H1
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:957)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:114)
at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:137)
at hudson.tasks.junit.JUnitResultArchiver.parseAndAttach(JUnitResultArchiver.java:167)
at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:52)
at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:25)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
... 4 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r399047446
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
Review comment:
dang.
ok. public for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r398998728
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
##########
@@ -317,7 +318,14 @@ protected boolean performCompaction(FileDetails fd, InternalScanner scanner, Cel
do {
hasMore = scanner.next(cells, scannerContext);
- now = EnvironmentEdgeManager.currentTime();
+ currentTime = EnvironmentEdgeManager.currentTime();
+ if (LOG.isDebugEnabled()) {
+ now = currentTime;
+ }
+ if (closeChecker.isClosedByTimeLimit(store, currentTime)) {
Review comment:
> This is a bug fix?
No, It is not a bug fix. I think it is `improvement`.
> s/isClosedByTimeLime/isTimeLimit/ ... and s/isClosedBySizeLiimt/isSizeLimit/
Okay.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605853080
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 41s | master passed |
| +1 :green_heart: | compile | 1m 19s | master passed |
| +1 :green_heart: | shadedjars | 6m 25s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 44s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 16s | the patch passed |
| +1 :green_heart: | javac | 1m 16s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 54s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 56s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 159m 30s | hbase-server in the patch failed. |
| | | 196m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7f379d59b25e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d4e020c3d |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/testReport/ |
| Max. process+thread count | 4086 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang edited a comment on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang edited a comment on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610743488
@saintstack All test is green. Could you check this PR?
and I want to apply this patch in branch-2.2. because I am preparing to make an HBase cluster using HBase-2.2.X. Does it make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606729395
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 51s | master passed |
| +1 :green_heart: | compile | 1m 22s | master passed |
| +1 :green_heart: | shadedjars | 6m 28s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 54s | the patch passed |
| +1 :green_heart: | compile | 1m 25s | the patch passed |
| +1 :green_heart: | javac | 1m 25s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 7m 20s | hbase-server in the patch failed. |
| | | 45m 13s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 56d22cbc435f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / fded2b9ddc |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/testReport/ |
| Max. process+thread count | 729 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610152889
`shadedjars` was failed. I will check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606331232
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 19s | master passed |
| +1 :green_heart: | compile | 0m 56s | master passed |
| +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 11s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 43s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 65m 44s | hbase-server in the patch passed. |
| | | 91m 22s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 357c2ff054f5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 030e833dc9 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/testReport/ |
| Max. process+thread count | 5529 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606714510
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 56s | master passed |
| +1 :green_heart: | checkstyle | 1m 13s | master passed |
| +1 :green_heart: | spotbugs | 2m 8s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 0s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 10m 29s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 3m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 36m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux e42424bb938e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / fded2b9ddc |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396669677
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606334416
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 24s | master passed |
| +1 :green_heart: | compile | 1m 14s | master passed |
| +1 :green_heart: | shadedjars | 5m 52s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 1s | the patch passed |
| +1 :green_heart: | compile | 1m 8s | the patch passed |
| +1 :green_heart: | javac | 1m 8s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 46s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 70m 26s | hbase-server in the patch passed. |
| | | 102m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a79a66895bb5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 030e833dc9 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/testReport/ |
| Max. process+thread count | 6670 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597697662
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 12s | master passed |
| +1 :green_heart: | compile | 1m 0s | master passed |
| +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 38s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 10s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 35s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 5m 51s | hbase-server in the patch failed. |
| | | 33m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a66f6497d529 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/testReport/ |
| Max. process+thread count | 1018 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610743488
@saintstack All test is green. Could you check this PR?
and I want to apply this patch in branch-2.2. because I am preparing to make an HBase cluster using HBase-2.2.4. Does it make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396671133
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
##########
@@ -317,7 +318,14 @@ protected boolean performCompaction(FileDetails fd, InternalScanner scanner, Cel
do {
hasMore = scanner.next(cells, scannerContext);
- now = EnvironmentEdgeManager.currentTime();
+ currentTime = EnvironmentEdgeManager.currentTime();
+ if (LOG.isDebugEnabled()) {
+ now = currentTime;
+ }
+ if (closeChecker.isClosedByTimeLimit(store, currentTime)) {
Review comment:
This is a bug fix?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610742359
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 55s | master passed |
| +1 :green_heart: | compile | 0m 55s | master passed |
| +1 :green_heart: | shadedjars | 4m 57s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 36s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 169m 51s | hbase-server in the patch passed. |
| | | 197m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c2be8903e7d8 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 80d4a09ddc |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/testReport/ |
| Max. process+thread count | 3617 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-609505078
@saintstack May I ask you to confirm it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r398997341
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
+ this.closeCheckSizeLimit = conf.getInt(SIZE_LIMIT_KEY, 10 * 1000 * 1000 /* 10 MB */);
+ this.closeCheckTimeLimit = conf.getLong(TIME_LIMIT_KEY, 10 * 1000L /* 10 s */);
+ this.bytesWrittenProgressForCloseCheck = 0;
+ this.lastCloseCheckMillis = currentTime;
+ }
+
+ public boolean isClosedBySizeLimit(Store store, long bytesWritten) {
+ if (closeCheckSizeLimit <= 0) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck += bytesWritten;
+ if (bytesWrittenProgressForCloseCheck <= closeCheckSizeLimit) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck = 0;
+ return !store.areWritesEnabled();
+ }
+
+ public boolean isClosedByTimeLimit(Store store, long now) {
+ if (closeCheckTimeLimit <= 0) {
+ return false;
+ }
+
+ final long elapsedMillis = now - lastCloseCheckMillis;
+ if (elapsedMillis <= closeCheckTimeLimit) {
+ return false;
+ }
+
+ lastCloseCheckMillis = now;
Review comment:
CloseChecker is created and used in `performCompaction`.
```java
protected boolean performCompaction() throws IOException {
CloseChecker closeChecker = new CloseChecker(conf, currentTime);
try {
do {
// ...
}
}
```
I think unuse `volatile` is okay.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610156045
The failure of `shadedjars` is due to rat plugins.
They show files with unapproved licenses:
```
dev-support/HOW_TO_YETUS_LOCAL.md
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610126881
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 45s | master passed |
| +1 :green_heart: | checkstyle | 1m 19s | master passed |
| +1 :green_heart: | spotbugs | 2m 9s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 24s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 17s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 18s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 37m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 4a72cd38c9b3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 15b94cc306 |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605742019
retest build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610732020
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 1s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 21s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 55s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 116m 1s | hbase-server in the patch passed. |
| | | 148m 21s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 620feb254aa1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 80d4a09ddc |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/testReport/ |
| Max. process+thread count | 4165 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597702779
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 36s | master passed |
| +1 :green_heart: | checkstyle | 1m 12s | master passed |
| +0 :ok: | spotbugs | 4m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 3m 1s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 4s | the patch passed |
| -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 220 unchanged - 0 fixed = 221 total (was 220) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 16m 18s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | findbugs | 2m 59s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 43m 3s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 427906d053d4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396672707
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
+ this.closeCheckSizeLimit = conf.getInt(SIZE_LIMIT_KEY, 10 * 1000 * 1000 /* 10 MB */);
+ this.closeCheckTimeLimit = conf.getLong(TIME_LIMIT_KEY, 10 * 1000L /* 10 s */);
+ this.bytesWrittenProgressForCloseCheck = 0;
+ this.lastCloseCheckMillis = currentTime;
+ }
+
+ public boolean isClosedBySizeLimit(Store store, long bytesWritten) {
Review comment:
s/isClosedBySizeLimit/isSizeLimit/g
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396669282
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
+ this.closeCheckSizeLimit = conf.getInt(SIZE_LIMIT_KEY, 10 * 1000 * 1000 /* 10 MB */);
+ this.closeCheckTimeLimit = conf.getLong(TIME_LIMIT_KEY, 10 * 1000L /* 10 s */);
+ this.bytesWrittenProgressForCloseCheck = 0;
+ this.lastCloseCheckMillis = currentTime;
+ }
+
+ public boolean isClosedBySizeLimit(Store store, long bytesWritten) {
+ if (closeCheckSizeLimit <= 0) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck += bytesWritten;
+ if (bytesWrittenProgressForCloseCheck <= closeCheckSizeLimit) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck = 0;
+ return !store.areWritesEnabled();
+ }
+
+ public boolean isClosedByTimeLimit(Store store, long now) {
+ if (closeCheckTimeLimit <= 0) {
+ return false;
+ }
+
+ final long elapsedMillis = now - lastCloseCheckMillis;
+ if (elapsedMillis <= closeCheckTimeLimit) {
+ return false;
+ }
+
+ lastCloseCheckMillis = now;
Review comment:
Single thread in here? Otherwise, perhaps this should be volatile?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396671715
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
##########
@@ -317,7 +318,14 @@ protected boolean performCompaction(FileDetails fd, InternalScanner scanner, Cel
do {
hasMore = scanner.next(cells, scannerContext);
- now = EnvironmentEdgeManager.currentTime();
+ currentTime = EnvironmentEdgeManager.currentTime();
+ if (LOG.isDebugEnabled()) {
+ now = currentTime;
+ }
+ if (closeChecker.isClosedByTimeLimit(store, currentTime)) {
Review comment:
s/isClosedByTimeLime/isTimeLimit/ ... and s/isClosedBySizeLiimt/isSizeLimit/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605793407
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 59s | master passed |
| +1 :green_heart: | checkstyle | 1m 20s | master passed |
| +1 :green_heart: | spotbugs | 2m 11s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 31s | the patch passed |
| -0 :warning: | checkstyle | 1m 18s | hbase-server: The patch generated 1 new + 213 unchanged - 0 fixed = 214 total (was 213) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 30s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 38m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux ad1626a69c66 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d4e020c3d |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r398998333
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
Review comment:
I think it would be good to use `package local`.
However, The package of `FaultyMobStoreCompactor` is not the same as `CloseChecker`.
The package of `FaultyMobStoreCompactor` is `org.apache.hadoop.hbase.mob`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606302459
Let me retry the run here too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606319825
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 18s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 48s | master passed |
| +1 :green_heart: | checkstyle | 1m 56s | master passed |
| +1 :green_heart: | spotbugs | 3m 22s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 56s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 45s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 14m 39s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 44s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 53m 46s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux f9110250cbf5 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 030e833dc9 |
| Max. process+thread count | 84 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597785047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 53s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 36s | master passed |
| +1 :green_heart: | compile | 1m 52s | master passed |
| +1 :green_heart: | shadedjars | 8m 2s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 1m 6s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 3s | the patch passed |
| +1 :green_heart: | compile | 1m 46s | the patch passed |
| +1 :green_heart: | javac | 1m 46s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 4s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 1m 0s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 79m 47s | hbase-server in the patch failed. |
| | | 125m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 55016c037132 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/testReport/ |
| Max. process+thread count | 8875 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605952313
@saintstack I fixed checkstyle error. But I don't know why unit was failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396673090
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
+ this.closeCheckSizeLimit = conf.getInt(SIZE_LIMIT_KEY, 10 * 1000 * 1000 /* 10 MB */);
+ this.closeCheckTimeLimit = conf.getLong(TIME_LIMIT_KEY, 10 * 1000L /* 10 s */);
+ this.bytesWrittenProgressForCloseCheck = 0;
+ this.lastCloseCheckMillis = currentTime;
+ }
+
+ public boolean isClosedBySizeLimit(Store store, long bytesWritten) {
+ if (closeCheckSizeLimit <= 0) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck += bytesWritten;
+ if (bytesWrittenProgressForCloseCheck <= closeCheckSizeLimit) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck = 0;
+ return !store.areWritesEnabled();
+ }
+
+ public boolean isClosedByTimeLimit(Store store, long now) {
Review comment:
s/isClosedByTimeLimit/isTimeLimit/g
... Class name provides the context ... i.e. that it is about close.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
busbey commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-605784039
I started the qabot again
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-604783606
@saintstack Thank you for your review. I reflected your feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang commented on issue #1274: HBASE-23968 Periodically
check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606339562
@saintstack Thank you!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610167040
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 52s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 36s | master passed |
| +1 :green_heart: | compile | 1m 7s | master passed |
| -1 :x: | shadedjars | 0m 19s | branch has 7 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 58s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 177m 27s | hbase-server in the patch passed. |
| | | 200m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b73f07206bc3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 15b94cc306 |
| Default Java | 1.8.0_232 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/testReport/ |
| Max. process+thread count | 3398 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/6/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-606765991
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 15s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| +1 :green_heart: | shadedjars | 5m 42s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 53s | the patch passed |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 37s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 65m 44s | hbase-server in the patch passed. |
| | | 95m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b6aa7afb856c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / fded2b9ddc |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/testReport/ |
| Max. process+thread count | 6044 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1274:
HBASE-23968 Periodically check whether a system stop is requested in
compaction by time.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#discussion_r396672413
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CloseChecker.java
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Check periodically to see if a system stop is requested
+ */
+@InterfaceAudience.Private
+public class CloseChecker {
+ public static final String SIZE_LIMIT_KEY = "hbase.hstore.close.check.interval";
+ public static final String TIME_LIMIT_KEY = "hbase.hstore.close.check.time.interval";
+
+ private final int closeCheckSizeLimit;
+ private final long closeCheckTimeLimit;
+
+ private long bytesWrittenProgressForCloseCheck;
+ private long lastCloseCheckMillis;
+
+ public CloseChecker(Configuration conf, long currentTime) {
+ this.closeCheckSizeLimit = conf.getInt(SIZE_LIMIT_KEY, 10 * 1000 * 1000 /* 10 MB */);
+ this.closeCheckTimeLimit = conf.getLong(TIME_LIMIT_KEY, 10 * 1000L /* 10 s */);
+ this.bytesWrittenProgressForCloseCheck = 0;
+ this.lastCloseCheckMillis = currentTime;
+ }
+
+ public boolean isClosedBySizeLimit(Store store, long bytesWritten) {
+ if (closeCheckSizeLimit <= 0) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck += bytesWritten;
+ if (bytesWrittenProgressForCloseCheck <= closeCheckSizeLimit) {
+ return false;
+ }
+
+ bytesWrittenProgressForCloseCheck = 0;
+ return !store.areWritesEnabled();
+ }
+
+ public boolean isClosedByTimeLimit(Store store, long now) {
+ if (closeCheckTimeLimit <= 0) {
+ return false;
+ }
+
+ final long elapsedMillis = now - lastCloseCheckMillis;
+ if (elapsedMillis <= closeCheckTimeLimit) {
+ return false;
+ }
+
+ lastCloseCheckMillis = now;
Review comment:
Better if can do w/o volatile (I think compactor will only have one driving thread...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mwkang edited a comment on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
mwkang edited a comment on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610743488
@saintstack All test is green. Could you check this PR?
and I want to apply this patch in branch-2.2, branch-1.x. because I am preparing to make an HBase cluster using HBase-2.2.X and I operate HBase cluster Using HBase-1.x. Does it make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-610703875
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 43s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 44s | master passed |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| +1 :green_heart: | spotbugs | 2m 18s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 36s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 18s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 23s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 2m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 40m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux c9c4783083fb 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 80d4a09ddc |
| Max. process+thread count | 83 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/7/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597700824
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 35s | master passed |
| +1 :green_heart: | compile | 1m 13s | master passed |
| +1 :green_heart: | shadedjars | 5m 38s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 6s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 7m 10s | hbase-server in the patch failed. |
| | | 39m 24s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 595f22157a86 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/testReport/ |
| Max. process+thread count | 935 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1274: HBASE-23968
Periodically check whether a system stop is requested in compaction by time.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1274: HBASE-23968 Periodically check whether a system stop is requested in compaction by time.
URL: https://github.com/apache/hbase/pull/1274#issuecomment-597741345
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 35s | master passed |
| +1 :green_heart: | checkstyle | 1m 12s | master passed |
| +0 :ok: | spotbugs | 4m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 2m 53s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 3s | the patch passed |
| -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 220 unchanged - 0 fixed = 221 total (was 220) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 16m 3s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | findbugs | 2m 59s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 42m 24s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1274 |
| Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 431d567b990e 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / d019fd2eed |
| findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1274/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services