You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/04/01 15:34:35 UTC
[GitHub] [pinot] richardstartin opened a new pull request #8457: add tracing SPI
richardstartin opened a new pull request #8457:
URL: https://github.com/apache/pinot/pull/8457
This is a straw man proposal for a tracing API which allows detailed capture of operator statistics, far beyond execution time. The default tracing implementation is delegated to by the tracing SPI which can be overridden at startup.
`Tracer` has two operations:
1. Register the requestId if tracing is enabled, the tracing implementation is responsible for propagating this to query threads. The default implementation is privileged in its use of `TraceCallable` and `TraceRunnable` but third part implementations will use class transformation to add a `requestId` field to `FutureTask`. It is also responsible for maintaining lineage between parent and child spans, stack maintenance etc..
2. Start an operator span.
Operator spans are closable, and are completed when closed.
`OperatorInvocationTrace` will be passed into block evaluation, and various fields can be recorded into it:
```java
public interface OperatorInvocationTrace {
/**
* Sets the class of the operator. This allows various class-level properties
* to be interrogated and cached in a {@see ClassValue}.
* @param operator the class of the operator
*/
void setOperatorClass(Class<?> operator);
/**
* Sets the number of docs scanned by the operator.
* @param docsScanned how many docs were scanned.
*/
void setDocsScanned(long docsScanned);
/**
* Sets the number of bytes scanned by the operator if this is possible to compute.
* @param bytesScanned the number of bytes scanned
*/
void setBytesProcessed(long bytesScanned);
/**
* If the operator is a filter, determines the filter type (scan or index) and the predicate type
* @param filterType SCAN or INDEX
* @param predicateType e.g. BETWEEN, REGEXP_LIKE
*/
void setFilterType(FilterType filterType, String predicateType);
/**
* The phase of the query
* @param phase the phase
*/
void setPhase(Phase phase);
/**
* Records whether type transformation took place during the operator's invocation and what the types were
* @param inputDataType the input data type
* @param outputDataType the output data type
*/
void setDataTypes(FieldSpec.DataType inputDataType, FieldSpec.DataType outputDataType);
/**
* Records the range of docIds during the operator invocation. This is useful for implicating a range of records
* in a slow operator invocation.
* @param firstDocId the first docId in the block
* @param lastDocId the last docId in the block
*/
void setDocIdRange(int firstDocId, int lastDocId);
/**
* If known, record the cardinality of the column within the segment this operator invoked on
* @param cardinality the number of distinct values
*/
void setColumnCardinality(int cardinality);
}
```
The default implementation records none of these. Operator implementations will need to be modified to record values into the `OperatorInvocationTrace`. Dead code elimination is relied upon to eliminate overhead when these values are written to default implementations of `OperatorInvocationTrace`. These implementations will not be modified before this SPI is agreed to.
The `Tracer` does not need to attach trace information to the output request, and where the trace information goes is implementation defined; it may output it to a file, an in-memory circular buffer which can be dumped on demand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #8457: add tracing SPI
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #8457:
URL: https://github.com/apache/pinot/pull/8457#issuecomment-1086112645
# [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#8457](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9bb3c2b) into [master](https://codecov.io/gh/apache/pinot/commit/e7a2f5b2c08b75a718bf2619dc9abe51a07736ed?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e7a2f5b) will **decrease** coverage by `3.65%`.
> The diff coverage is `32.00%`.
```diff
@@ Coverage Diff @@
## master #8457 +/- ##
============================================
- Coverage 70.68% 67.02% -3.66%
+ Complexity 4282 4197 -85
============================================
Files 1663 1265 -398
Lines 87288 63774 -23514
Branches 13210 10011 -3199
============================================
- Hits 61697 42744 -18953
+ Misses 21305 17956 -3349
+ Partials 4286 3074 -1212
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `67.02% <32.00%> (-0.03%)` | :arrow_down: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...core/query/executor/ServerQueryExecutorV1Impl.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9TZXJ2ZXJRdWVyeUV4ZWN1dG9yVjFJbXBsLmphdmE=) | `60.09% <0.00%> (-22.48%)` | :arrow_down: |
| [...in/java/org/apache/pinot/spi/trace/FilterType.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvRmlsdGVyVHlwZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...rc/main/java/org/apache/pinot/spi/trace/Phase.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvUGhhc2UuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...rg/apache/pinot/core/util/trace/DefaultTracer.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL3RyYWNlL0RlZmF1bHRUcmFjZXIuamF2YQ==) | `24.00% <24.00%> (ø)` | |
| [.../main/java/org/apache/pinot/spi/trace/Tracing.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvVHJhY2luZy5qYXZh) | `58.33% <58.33%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `66.66% <100.00%> (+28.20%)` | :arrow_up: |
| [...va/org/apache/pinot/core/routing/RoutingTable.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yb3V0aW5nL1JvdXRpbmdUYWJsZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...va/org/apache/pinot/common/config/NettyConfig.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL05ldHR5Q29uZmlnLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [625 more](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [e7a2f5b...9bb3c2b](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on pull request #8457: add tracing SPI
Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #8457:
URL: https://github.com/apache/pinot/pull/8457#issuecomment-1086133603
> Huge +1
>
> Thanks @richardstartin . I will review the approach today. Meanwhile, wanted to check on one thing:
>
> We had added the threadCpuTime metric (measured at the server combine level). As part of the [next work for EXPLAIN PLAN](https://docs.google.com/document/d/1lBMGcR65JY_8iIUCA1iCazzn0LzTVxMJlPzFIsZ7TRk/edit#heading=h.kwlkkfaymsj0), we wanted to push down the measurement of this metric (or at least evaluate if it is feasible to do so) and other metrics at the lower / leaf operator level as well with the goal of creating a QueryProfile / Operator level stats.
>
> I am guessing the new abstraction proposed in this PR will allow us to do so. Just want to make sure it won't prevent us from doing so and this can be further extended
The idea is to be able to implement that sort of thing at the operator layer, yes. The main goal is to not need to agree what gets recorded in a span or how. This design also leaves context propagation up to the implementor, who can choose between bytecode instrumentation (which allows propagation of arbitrary information) and the existing, rather limited, mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #8457: add tracing SPI
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8457:
URL: https://github.com/apache/pinot/pull/8457#issuecomment-1086112645
# [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#8457](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9bb3c2b) into [master](https://codecov.io/gh/apache/pinot/commit/e7a2f5b2c08b75a718bf2619dc9abe51a07736ed?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e7a2f5b) will **decrease** coverage by `8.41%`.
> The diff coverage is `32.00%`.
```diff
@@ Coverage Diff @@
## master #8457 +/- ##
============================================
- Coverage 70.68% 62.26% -8.42%
+ Complexity 4282 4197 -85
============================================
Files 1663 1655 -8
Lines 87288 86973 -315
Branches 13210 13173 -37
============================================
- Hits 61697 54153 -7544
- Misses 21305 28809 +7504
+ Partials 4286 4011 -275
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `27.05% <14.00%> (-0.05%)` | :arrow_down: |
| integration2 | `?` | |
| unittests1 | `67.02% <32.00%> (-0.03%)` | :arrow_down: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...core/query/executor/ServerQueryExecutorV1Impl.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9TZXJ2ZXJRdWVyeUV4ZWN1dG9yVjFJbXBsLmphdmE=) | `80.73% <0.00%> (-1.84%)` | :arrow_down: |
| [...in/java/org/apache/pinot/spi/trace/FilterType.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvRmlsdGVyVHlwZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...rc/main/java/org/apache/pinot/spi/trace/Phase.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvUGhhc2UuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...rg/apache/pinot/core/util/trace/DefaultTracer.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL3RyYWNlL0RlZmF1bHRUcmFjZXIuamF2YQ==) | `24.00% <24.00%> (ø)` | |
| [.../main/java/org/apache/pinot/spi/trace/Tracing.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvVHJhY2luZy5qYXZh) | `58.33% <58.33%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `100.00% <100.00%> (+61.53%)` | :arrow_up: |
| [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...t/core/plan/StreamingInstanceResponsePlanNode.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9wbGFuL1N0cmVhbWluZ0luc3RhbmNlUmVzcG9uc2VQbGFuTm9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/controller/recommender/io/ConfigManager.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9pby9Db25maWdNYW5hZ2VyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/operator/streaming/StreamingResponseUtils.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9zdHJlYW1pbmcvU3RyZWFtaW5nUmVzcG9uc2VVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [289 more](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [e7a2f5b...9bb3c2b](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #8457: add tracing SPI
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8457:
URL: https://github.com/apache/pinot/pull/8457#issuecomment-1086112645
# [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#8457](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b21cc5f) into [master](https://codecov.io/gh/apache/pinot/commit/e7a2f5b2c08b75a718bf2619dc9abe51a07736ed?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e7a2f5b) will **decrease** coverage by `0.04%`.
> The diff coverage is `32.65%`.
```diff
@@ Coverage Diff @@
## master #8457 +/- ##
============================================
- Coverage 70.68% 70.63% -0.05%
- Complexity 4282 4283 +1
============================================
Files 1663 1667 +4
Lines 87288 87326 +38
Branches 13210 13211 +1
============================================
- Hits 61697 61687 -10
- Misses 21305 21355 +50
+ Partials 4286 4284 -2
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `27.07% <14.28%> (-0.03%)` | :arrow_down: |
| integration2 | `25.97% <14.28%> (-0.03%)` | :arrow_down: |
| unittests1 | `67.03% <32.65%> (-0.02%)` | :arrow_down: |
| unittests2 | `14.15% <0.00%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...core/query/executor/ServerQueryExecutorV1Impl.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9TZXJ2ZXJRdWVyeUV4ZWN1dG9yVjFJbXBsLmphdmE=) | `86.69% <0.00%> (+4.12%)` | :arrow_up: |
| [...in/java/org/apache/pinot/spi/trace/FilterType.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvRmlsdGVyVHlwZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...rc/main/java/org/apache/pinot/spi/trace/Phase.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvUGhhc2UuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...rg/apache/pinot/core/util/trace/DefaultTracer.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL3RyYWNlL0RlZmF1bHRUcmFjZXIuamF2YQ==) | `25.00% <25.00%> (ø)` | |
| [.../main/java/org/apache/pinot/spi/trace/Tracing.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvVHJhY2luZy5qYXZh) | `58.33% <58.33%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `100.00% <100.00%> (+61.53%)` | :arrow_up: |
| [...r/api/resources/WebApplicationExceptionMapper.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1dlYkFwcGxpY2F0aW9uRXhjZXB0aW9uTWFwcGVyLmphdmE=) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [...ontroller/api/resources/ServerTableSizeReader.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1NlcnZlclRhYmxlU2l6ZVJlYWRlci5qYXZh) | `83.33% <0.00%> (-16.67%)` | :arrow_down: |
| [.../predicate/NotEqualsPredicateEvaluatorFactory.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvcHJlZGljYXRlL05vdEVxdWFsc1ByZWRpY2F0ZUV2YWx1YXRvckZhY3RvcnkuamF2YQ==) | `66.17% <0.00%> (-11.77%)` | :arrow_down: |
| [...a/org/apache/pinot/common/utils/ServiceStatus.java](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvU2VydmljZVN0YXR1cy5qYXZh) | `60.00% <0.00%> (-7.15%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/pinot/pull/8457/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [e7a2f5b...b21cc5f](https://codecov.io/gh/apache/pinot/pull/8457?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on pull request #8457: add tracing SPI
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on pull request #8457:
URL: https://github.com/apache/pinot/pull/8457#issuecomment-1086097380
Huge +1
Thanks @richardstartin . I will review the approach today. Meanwhile, wanted to check on one thing:
We had added the threadCpuTime metric (measured at the server combine level). As part of the [next work for EXPLAIN PLAN](https://docs.google.com/document/d/1lBMGcR65JY_8iIUCA1iCazzn0LzTVxMJlPzFIsZ7TRk/edit#heading=h.kwlkkfaymsj0), we wanted to push down the measurement of this metric (or at least evaluate if it is feasible to do so) and other metrics at the lower / leaf operator level as well with the goal of creating a QueryProfile / Operator level stats.
I am guessing the new abstraction proposed in this PR will allow us to do so. Just want to make sure it won't prevent us from doing so and this can be further extended
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org