You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/04/28 15:21:17 UTC
[GitHub] [hbase] ndimiduk opened a new pull request, #4381: HBASE-26986 Trace Master Procedure Execution
ndimiduk opened a new pull request, #4381:
URL: https://github.com/apache/hbase/pull/4381
This one adds a tracing context to the Master's ProcedureV2 execution. This ties all the spans that result from a procedure execution back to a procedure by name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4381: HBASE-26986 Trace a one-shot execution of a Master procedure
Posted by GitBox <gi...@apache.org>.
ndimiduk merged PR #4381:
URL: https://github.com/apache/hbase/pull/4381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4381:
URL: https://github.com/apache/hbase/pull/4381#discussion_r862836283
##########
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java:
##########
@@ -1971,27 +2002,7 @@ public void run() {
continue;
}
this.activeProcedure = proc;
- int activeCount = activeExecutorCount.incrementAndGet();
- int runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Execute pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- executionStartTime.set(EnvironmentEdgeManager.currentTime());
- IdLock.Entry lockEntry = procExecutionLock.getLockEntry(proc.getProcId());
- try {
- executeProcedure(proc);
- } catch (AssertionError e) {
- LOG.info("ASSERT pid=" + proc.getProcId(), e);
- throw e;
- } finally {
- procExecutionLock.releaseLockEntry(lockEntry);
- activeCount = activeExecutorCount.decrementAndGet();
- runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Halt pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- this.activeProcedure = null;
- lastUpdate = EnvironmentEdgeManager.currentTime();
- executionStartTime.set(Long.MAX_VALUE);
- }
+ lastUpdate = TraceUtil.trace(this::runProcedure, new ProcedureSpanBuilder(proc));
Review Comment:
Something like 'Trace a one-shot execution of a procedure' seems more accurate :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1112344031
This gives us nicely contained traces like this,
```
├─ [spanId=52131d42d4b9b99a,name=TransitRegionStateProcedure table=testCreateOpenDeleteTableSpans, region=dcbb606f964a33df720cae68ae9d44b3, UNASSIGN,hasEnded=true]
└─ [spanId=b110180fbf63f086,name=PUT hbase:meta,hasEnded=true]
└─ [spanId=d68bcb4131ebe421,name=AsyncRegionLocator.getRegionLocation,hasEnded=true]
├─ [spanId=f3dec62bf805b93d,name=hbase.pb.ClientService/Mutate,hasEnded=true]
└─ [spanId=b919b8289e36f306,name=RpcServer.process,hasEnded=true]
└─ [spanId=458fd550bcf0fc10,name=hbase.pb.ClientService/Mutate,hasEnded=true]
└─ [spanId=bba75977ca158a3e,name=Region.put,hasEnded=true]
└─ [spanId=48e63e8dd8cedce9,name=WAL.sync,hasEnded=true]
├─ [spanId=f2545d1cdd8df88c,name=WAL.appendData,hasEnded=true]
├─ [spanId=5245192c950f563b,name=Region.getRowLock,hasEnded=true]
├─ [spanId=85d7316b561ca0fc,name=WAL.appendData,hasEnded=true]
├─ [spanId=95d636764e433992,name=Region.getRowLock,hasEnded=true]
├─ [spanId=9360c4de11428864,name=WAL.sync,hasEnded=true]
├─ [spanId=7a78b0d2ef332573,name=Region.put,hasEnded=true]
└─ [spanId=99dbb2e60aa47b12,name=WAL.sync,hasEnded=true]
├─ [spanId=ba71434842eb98db,name=Region.getRowLock,hasEnded=true]
├─ [spanId=8bfd48b9277978b7,name=WAL.appendData,hasEnded=true]
├─ [spanId=8f0717e91a8852c9,name=Region.getRowLock,hasEnded=true]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1116269993
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 3s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 22s | master passed |
| +1 :green_heart: | compile | 1m 11s | master passed |
| +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 12s | the patch passed |
| +1 :green_heart: | javac | 1m 12s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 41s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 1s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 11s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 192m 28s | hbase-server in the patch passed. |
| | | 213m 56s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 33109de0de54 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 404e1d6870 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/testReport/ |
| Max. process+thread count | 2680 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1112557219
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 47s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 17s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 3m 56s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 28s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 184m 51s | hbase-server in the patch passed. |
| | | 209m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d6351d82edd6 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ec78c2677b |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/testReport/ |
| Max. process+thread count | 2931 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4381:
URL: https://github.com/apache/hbase/pull/4381#discussion_r862701825
##########
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java:
##########
@@ -1971,27 +2002,7 @@ public void run() {
continue;
}
this.activeProcedure = proc;
- int activeCount = activeExecutorCount.incrementAndGet();
- int runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Execute pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- executionStartTime.set(EnvironmentEdgeManager.currentTime());
- IdLock.Entry lockEntry = procExecutionLock.getLockEntry(proc.getProcId());
- try {
- executeProcedure(proc);
- } catch (AssertionError e) {
- LOG.info("ASSERT pid=" + proc.getProcId(), e);
- throw e;
- } finally {
- procExecutionLock.releaseLockEntry(lockEntry);
- activeCount = activeExecutorCount.decrementAndGet();
- runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Halt pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- this.activeProcedure = null;
- lastUpdate = EnvironmentEdgeManager.currentTime();
- executionStartTime.set(Long.MAX_VALUE);
- }
+ lastUpdate = TraceUtil.trace(this::runProcedure, new ProcedureSpanBuilder(proc));
Review Comment:
This implementation is very naive. There is no relationship between parent and child procedure. There is no concept of tying together multiple suspend/resume executions of the same procedure ID. It's merely a way to recognize that all the resulting RPCs are tied back to a procedure by ID. We can explore those additional features, and I agree that effort should include a lot more thought.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1112346984
FYI, I make only the most trivial effort to deconstruct the details of a procedure onto span attributes. I could see this becoming much more involved, by translating many of the internal details of a specific type of procedure into attributes that decorate the span.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1116261335
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 46s | master passed |
| +1 :green_heart: | compile | 1m 22s | master passed |
| +1 :green_heart: | shadedjars | 3m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 30s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 22s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 19s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 183m 43s | hbase-server in the patch passed. |
| | | 206m 16s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ec0eb11620ca 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 404e1d6870 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/testReport/ |
| Max. process+thread count | 2632 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4381:
URL: https://github.com/apache/hbase/pull/4381#discussion_r862705449
##########
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java:
##########
@@ -1971,27 +2002,7 @@ public void run() {
continue;
}
this.activeProcedure = proc;
- int activeCount = activeExecutorCount.incrementAndGet();
- int runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Execute pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- executionStartTime.set(EnvironmentEdgeManager.currentTime());
- IdLock.Entry lockEntry = procExecutionLock.getLockEntry(proc.getProcId());
- try {
- executeProcedure(proc);
- } catch (AssertionError e) {
- LOG.info("ASSERT pid=" + proc.getProcId(), e);
- throw e;
- } finally {
- procExecutionLock.releaseLockEntry(lockEntry);
- activeCount = activeExecutorCount.decrementAndGet();
- runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Halt pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- this.activeProcedure = null;
- lastUpdate = EnvironmentEdgeManager.currentTime();
- executionStartTime.set(Long.MAX_VALUE);
- }
+ lastUpdate = TraceUtil.trace(this::runProcedure, new ProcedureSpanBuilder(proc));
Review Comment:
I can update the JIRA title to make it sound less thorough in its scope, if that helps at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1116074764
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 51s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 29s | master passed |
| +1 :green_heart: | compile | 3m 5s | master passed |
| +1 :green_heart: | checkstyle | 0m 46s | master passed |
| +1 :green_heart: | spotbugs | 2m 5s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 6s | the patch passed |
| +1 :green_heart: | compile | 3m 3s | the patch passed |
| +1 :green_heart: | javac | 3m 3s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 45s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 32s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 35m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2b57940b6a3d 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 404e1d6870 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace a one-shot execution of a Master procedure
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1126218563
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 49s | master passed |
| +1 :green_heart: | compile | 1m 20s | master passed |
| +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 35s | the patch passed |
| +1 :green_heart: | compile | 1m 22s | the patch passed |
| +1 :green_heart: | javac | 1m 22s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 22s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 13s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 187m 9s | hbase-server in the patch passed. |
| | | 210m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c6b323ee4d05 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ba7ef0216a |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/testReport/ |
| Max. process+thread count | 2469 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4381:
URL: https://github.com/apache/hbase/pull/4381#discussion_r862346496
##########
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java:
##########
@@ -1971,27 +2002,7 @@ public void run() {
continue;
}
this.activeProcedure = proc;
- int activeCount = activeExecutorCount.incrementAndGet();
- int runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Execute pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- executionStartTime.set(EnvironmentEdgeManager.currentTime());
- IdLock.Entry lockEntry = procExecutionLock.getLockEntry(proc.getProcId());
- try {
- executeProcedure(proc);
- } catch (AssertionError e) {
- LOG.info("ASSERT pid=" + proc.getProcId(), e);
- throw e;
- } finally {
- procExecutionLock.releaseLockEntry(lockEntry);
- activeCount = activeExecutorCount.decrementAndGet();
- runningCount = store.setRunningProcedureCount(activeCount);
- LOG.trace("Halt pid={} runningCount={}, activeCount={}", proc.getProcId(),
- runningCount, activeCount);
- this.activeProcedure = null;
- lastUpdate = EnvironmentEdgeManager.currentTime();
- executionStartTime.set(Long.MAX_VALUE);
- }
+ lastUpdate = TraceUtil.trace(this::runProcedure, new ProcedureSpanBuilder(proc));
Review Comment:
In a procedure's whole life time, it could reach here multiple times. For example, if it schedules some sub procedures, it will suspend itself for a while until all the sub procedures are finished, it will recover the execution. So I think if we want to trace procedure execution, we need to discuss more, and better to have a design doc first. It is not very straight forward, as how to define the relationship between parent procedure and sub procedure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace a one-shot execution of a Master procedure
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1126043634
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 22s | master passed |
| +1 :green_heart: | compile | 3m 10s | master passed |
| +1 :green_heart: | checkstyle | 0m 48s | master passed |
| +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 14s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 6s | the patch passed |
| +1 :green_heart: | compile | 3m 15s | the patch passed |
| +1 :green_heart: | javac | 3m 15s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 47s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 12m 1s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 29s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 38m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 8cb566fcba12 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ba7ef0216a |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace a one-shot execution of a Master procedure
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1126212307
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 50s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 46s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | shadedjars | 3m 55s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 24s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 183m 43s | hbase-server in the patch passed. |
| | | 206m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 440f4892605b 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ba7ef0216a |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/testReport/ |
| Max. process+thread count | 2890 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1112394771
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 22s | master passed |
| +1 :green_heart: | compile | 3m 17s | master passed |
| +1 :green_heart: | checkstyle | 1m 3s | master passed |
| +1 :green_heart: | spotbugs | 2m 19s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 12s | the patch passed |
| +1 :green_heart: | compile | 3m 15s | the patch passed |
| +1 :green_heart: | javac | 3m 15s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 59s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 42s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 34s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
| | | 37m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0bafdb98bb37 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ec78c2677b |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1112556002
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 41s | master passed |
| +1 :green_heart: | compile | 1m 25s | master passed |
| +1 :green_heart: | shadedjars | 3m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 33s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 37s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 22s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 1m 13s | hbase-procedure in the patch passed. |
| +1 :green_heart: | unit | 185m 29s | hbase-server in the patch passed. |
| | | 208m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4381 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4fef55253de1 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ec78c2677b |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/testReport/ |
| Max. process+thread count | 2506 (vs. ulimit of 30000) |
| modules | C: hbase-client hbase-procedure hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4381/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4381: HBASE-26986 Trace Master Procedure Execution
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4381:
URL: https://github.com/apache/hbase/pull/4381#issuecomment-1116033984
Rebased across formatting and updated Jira Summary and commit title according to reviewer feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org