You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/12/15 22:47:33 UTC
[GitHub] [hbase] ndimiduk opened a new pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
ndimiduk opened a new pull request #3951:
URL: https://github.com/apache/hbase/pull/3951
For batch operations, collect and annotate the associated span with the set of all operations contained in the batch.
This is an HBase-specific extension that I am proposing. The idea here is to provide a single field containing all of the operations contained within an "envelope-type" operation. That field can then be indexed for query, allowing operators to easily find the operations that they seek. This is a project-specific proposal for a solution to the general issue I raised on open-telemetry/opentelemetry-specification#2202.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1006372730
> Let's wait on this change until either we receive clear direction from [open-telemetry/opentelemetry-specification#2202](https://github.com/open-telemetry/opentelemetry-specification/issues/2202) or I can show clear benefit in my end-to-end testing.
Seems no progress on the otel side? So maybe it is a chance for us to define the best practice of tracing multi/batch operations?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1020864102
> > Seems no progress on the otel side? So maybe it is a chance for us to define the best practice of tracing multi/batch operations?
>
> Still no comments from them. If you are still +1, @Apache9 , I'll rebase this commit and merge it. Lack of response from upstream isn't great though.
Just go ahead. I'm OK with the current approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1022559120
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 10s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 47s | master passed |
| +1 :green_heart: | compile | 1m 23s | master passed |
| +1 :green_heart: | shadedjars | 11m 38s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 21s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 44s | the patch passed |
| +1 :green_heart: | compile | 1m 21s | the patch passed |
| +1 :green_heart: | javac | 1m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 11m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 5s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 16s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 2m 25s | hbase-client in the patch passed. |
| | | 56m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux fd87ccc09f19 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7be9f26e40 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/testReport/ |
| Max. process+thread count | 242 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#discussion_r771735530
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/trace/TableOperationSpanBuilder.java
##########
@@ -82,6 +89,72 @@ public TableOperationSpanBuilder setOperation(final Operation operation) {
return this;
}
+ // `setContainerOperations` perform a recursive descent expansion of all the operations
+ // contained within the provided "batch" object.
+
+ public TableOperationSpanBuilder setContainerOperations(final RowMutations mutations) {
+ final Operation[] ops = mutations.getMutations()
+ .stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(final Row row) {
+ final Operation[] ops =
+ Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream())
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
+ final Collection<? extends Row> operations
+ ) {
+ final Operation[] ops = operations.stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ private static Set<Operation> unpackRowOperations(final Row row) {
+ final Set<Operation> ops = new HashSet<>();
+ if (row instanceof CheckAndMutate) {
+ final CheckAndMutate cam = (CheckAndMutate) row;
+ ops.addAll(unpackRowOperations(cam));
+ }
+ if (row instanceof RowMutations) {
+ final RowMutations mutations = (RowMutations) row;
+ ops.addAll(unpackRowOperations(mutations));
+ }
+ return ops;
+ }
+
+ private static Set<Operation> unpackRowOperations(final CheckAndMutate cam) {
+ final Set<Operation> ops = new HashSet<>();
+ final Operation op = valueFrom(cam.getAction());
+ switch (op) {
+ case BATCH:
+ case CHECK_AND_MUTATE:
+ ops.addAll(unpackRowOperations(cam.getAction()));
+ break;
+ default:
+ ops.add(op);
+ }
+ return ops;
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
Review comment:
That's an interesting idea! What do you have in mind, something like `db.hbase.container_operations.get.count` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-995295045
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 14s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 37s | master passed |
| +1 :green_heart: | compile | 0m 55s | master passed |
| +1 :green_heart: | shadedjars | 9m 11s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 20s | the patch passed |
| +1 :green_heart: | compile | 0m 52s | the patch passed |
| +1 :green_heart: | javac | 0m 52s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. |
| | | 37m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux fea62f726dbb 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8f5a12f794 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/testReport/ |
| Max. process+thread count | 227 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1022555312
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 24s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 43s | master passed |
| +1 :green_heart: | compile | 1m 56s | master passed |
| +1 :green_heart: | checkstyle | 0m 56s | master passed |
| +1 :green_heart: | spotbugs | 2m 2s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 19s | the patch passed |
| +1 :green_heart: | compile | 1m 54s | the patch passed |
| +1 :green_heart: | javac | 0m 51s | hbase-common in the patch passed. |
| +1 :green_heart: | javac | 1m 3s | hbase-client generated 0 new + 121 unchanged - 1 fixed = 121 total (was 122) |
| +1 :green_heart: | checkstyle | 0m 56s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 22m 27s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 9s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 52m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 60ac528f68bc 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7be9f26e40 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1020531596
> Seems no progress on the otel side? So maybe it is a chance for us to define the best practice of tracing multi/batch operations?
Still no comments from them. If you are still +1, @Apache9 , I'll rebase this commit and merge it. Lack of response from upstream isn't great though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1020531596
> Seems no progress on the otel side? So maybe it is a chance for us to define the best practice of tracing multi/batch operations?
Still no comments from them. If you are still +1, @Apache9 , I'll rebase this commit and merge it. Lack of response from upstream isn't great though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#discussion_r770366748
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/trace/TableOperationSpanBuilder.java
##########
@@ -82,6 +89,72 @@ public TableOperationSpanBuilder setOperation(final Operation operation) {
return this;
}
+ // `setContainerOperations` perform a recursive descent expansion of all the operations
+ // contained within the provided "batch" object.
+
+ public TableOperationSpanBuilder setContainerOperations(final RowMutations mutations) {
+ final Operation[] ops = mutations.getMutations()
+ .stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(final Row row) {
+ final Operation[] ops =
+ Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream())
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
+ final Collection<? extends Row> operations
+ ) {
+ final Operation[] ops = operations.stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ private static Set<Operation> unpackRowOperations(final Row row) {
+ final Set<Operation> ops = new HashSet<>();
+ if (row instanceof CheckAndMutate) {
+ final CheckAndMutate cam = (CheckAndMutate) row;
+ ops.addAll(unpackRowOperations(cam));
+ }
+ if (row instanceof RowMutations) {
+ final RowMutations mutations = (RowMutations) row;
+ ops.addAll(unpackRowOperations(mutations));
+ }
+ return ops;
+ }
+
+ private static Set<Operation> unpackRowOperations(final CheckAndMutate cam) {
+ final Set<Operation> ops = new HashSet<>();
+ final Operation op = valueFrom(cam.getAction());
+ switch (op) {
+ case BATCH:
+ case CHECK_AND_MUTATE:
+ ops.addAll(unpackRowOperations(cam.getAction()));
+ break;
+ default:
+ ops.add(op);
+ }
+ return ops;
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
Review comment:
Do we want to store a key value pair here to also reflect the cout for each operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1022544856
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 22s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 45s | master passed |
| +1 :green_heart: | compile | 0m 58s | master passed |
| +1 :green_heart: | shadedjars | 9m 31s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 24s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. |
| | | 38m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 70da4717816f 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7be9f26e40 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/testReport/ |
| Max. process+thread count | 220 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-995299179
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 3s | master passed |
| +1 :green_heart: | compile | 1m 45s | master passed |
| +1 :green_heart: | checkstyle | 0m 53s | master passed |
| +1 :green_heart: | spotbugs | 1m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 47s | the patch passed |
| +1 :green_heart: | compile | 1m 44s | the patch passed |
| +1 :green_heart: | javac | 0m 48s | hbase-common in the patch passed. |
| +1 :green_heart: | javac | 0m 56s | hbase-client generated 0 new + 121 unchanged - 1 fixed = 121 total (was 122) |
| +1 :green_heart: | checkstyle | 0m 51s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
| | | 46m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8f7684e027a7 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8f5a12f794 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#discussion_r771778363
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/trace/TableOperationSpanBuilder.java
##########
@@ -82,6 +89,72 @@ public TableOperationSpanBuilder setOperation(final Operation operation) {
return this;
}
+ // `setContainerOperations` perform a recursive descent expansion of all the operations
+ // contained within the provided "batch" object.
+
+ public TableOperationSpanBuilder setContainerOperations(final RowMutations mutations) {
+ final Operation[] ops = mutations.getMutations()
+ .stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(final Row row) {
+ final Operation[] ops =
+ Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream())
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
+ final Collection<? extends Row> operations
+ ) {
+ final Operation[] ops = operations.stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ private static Set<Operation> unpackRowOperations(final Row row) {
+ final Set<Operation> ops = new HashSet<>();
+ if (row instanceof CheckAndMutate) {
+ final CheckAndMutate cam = (CheckAndMutate) row;
+ ops.addAll(unpackRowOperations(cam));
+ }
+ if (row instanceof RowMutations) {
+ final RowMutations mutations = (RowMutations) row;
+ ops.addAll(unpackRowOperations(mutations));
+ }
+ return ops;
+ }
+
+ private static Set<Operation> unpackRowOperations(final CheckAndMutate cam) {
+ final Set<Operation> ops = new HashSet<>();
+ final Operation op = valueFrom(cam.getAction());
+ switch (op) {
+ case BATCH:
+ case CHECK_AND_MUTATE:
+ ops.addAll(unpackRowOperations(cam.getAction()));
+ break;
+ default:
+ ops.add(op);
+ }
+ return ops;
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
Review comment:
Just a simple idea, not sure what is the best way to implement it. Checked the code, seems not easy to have a key value structure for an attribute? If we just encode it with a String, like 'Put,1000', seems it will make searching this attribute a bit harder? But if we introduce another attribute to store the count, we need to make these two attributes aligned, which is also a pain...
Anyway, not a big problem for now, could consider it later when we learn opentelemetry more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-995296097
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 59s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 17s | master passed |
| +1 :green_heart: | compile | 0m 57s | master passed |
| +1 :green_heart: | shadedjars | 9m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 10s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 50s | hbase-client in the patch passed. |
| | | 39m 36s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 95845900ecb0 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8f5a12f794 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/testReport/ |
| Max. process+thread count | 217 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3951/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#issuecomment-1020864102
> > Seems no progress on the otel side? So maybe it is a chance for us to define the best practice of tracing multi/batch operations?
>
> Still no comments from them. If you are still +1, @Apache9 , I'll rebase this commit and merge it. Lack of response from upstream isn't great though.
Just go ahead. I'm OK with the current approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #3951:
URL: https://github.com/apache/hbase/pull/3951#discussion_r772563706
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/trace/TableOperationSpanBuilder.java
##########
@@ -82,6 +89,72 @@ public TableOperationSpanBuilder setOperation(final Operation operation) {
return this;
}
+ // `setContainerOperations` perform a recursive descent expansion of all the operations
+ // contained within the provided "batch" object.
+
+ public TableOperationSpanBuilder setContainerOperations(final RowMutations mutations) {
+ final Operation[] ops = mutations.getMutations()
+ .stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(final Row row) {
+ final Operation[] ops =
+ Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream())
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
+ final Collection<? extends Row> operations
+ ) {
+ final Operation[] ops = operations.stream()
+ .flatMap(row -> Stream.concat(Stream.of(valueFrom(row)), unpackRowOperations(row).stream()))
+ .toArray(Operation[]::new);
+ return setContainerOperations(ops);
+ }
+
+ private static Set<Operation> unpackRowOperations(final Row row) {
+ final Set<Operation> ops = new HashSet<>();
+ if (row instanceof CheckAndMutate) {
+ final CheckAndMutate cam = (CheckAndMutate) row;
+ ops.addAll(unpackRowOperations(cam));
+ }
+ if (row instanceof RowMutations) {
+ final RowMutations mutations = (RowMutations) row;
+ ops.addAll(unpackRowOperations(mutations));
+ }
+ return ops;
+ }
+
+ private static Set<Operation> unpackRowOperations(final CheckAndMutate cam) {
+ final Set<Operation> ops = new HashSet<>();
+ final Operation op = valueFrom(cam.getAction());
+ switch (op) {
+ case BATCH:
+ case CHECK_AND_MUTATE:
+ ops.addAll(unpackRowOperations(cam.getAction()));
+ break;
+ default:
+ ops.add(op);
+ }
+ return ops;
+ }
+
+ public TableOperationSpanBuilder setContainerOperations(
Review comment:
That's not a bad idea. I don't now how the telemetry services will handle these kinds of fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #3951: HBASE-26473 Introduce `db.hbase.container_operations` span attribute
Posted by GitBox <gi...@apache.org>.
ndimiduk merged pull request #3951:
URL: https://github.com/apache/hbase/pull/3951
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org