You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Akanksha-kedia (via GitHub)" <gi...@apache.org> on 2026/02/06 05:36:45 UTC

[PR] Add controller execution time tracking to query responses [pinot]

Akanksha-kedia opened a new pull request, #17647:
URL: https://github.com/apache/pinot/pull/17647

   - Add controllerExecutionTimeMs field to BrokerResponseNative
   - Include in JSON serialization order after timeUsedMs
   - Initialize to -1 to indicate when not applicable
   - Add getter/setter methods with proper JavaDoc
   
   This provides visibility into controller-level query performance for monitoring and debugging purposes.
   
   Instructions:
   1. The PR has to be tagged with at least one of the following labels (*):
      1. `feature`
      2. `bugfix`
      3. `performance`
      4. `ui`
      5. `backward-incompat`
      6. `release-notes` (**)
   2. Remove these instructions before publishing the PR.
    
   (*) Other labels to consider:
   - `testing`
   - `dependencies`
   - `docker`
   - `kubernetes`
   - `observability`
   - `security`
   - `code-style`
   - `extension-point`
   - `refactor`
   - `cleanup`
   
   (**) Use `release-notes` label for scenarios like:
   - New configuration options
   - Deprecation of configurations
   - Signature changes to public methods/interfaces
   - New plugins added or old plugins removed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "Akanksha-kedia (via GitHub)" <gi...@apache.org>.
Akanksha-kedia commented on PR #17647:
URL: https://github.com/apache/pinot/pull/17647#issuecomment-3902155557

   curl -X POST http://localhost:9001/sql -H "Content-Type: application/json" -d '{"sql":"SELECT COUNT(*) FROM baseballStats"}' | jq .
   
     % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                    Dload  Upload   Total   Spent    Left  Speed
   100  1610  100  1566  100    44  28979    814 --:--:-- --:--:-- --:--:-- 29814
   {
     "resultTable": {
       "dataSchema": {
         "columnNames": [
           "count(*)"
         ],
         "columnDataTypes": [
           "LONG"
         ]
       },
       "rows": [
         [
           97889
         ]
       ]
     },
     "numRowsResultSet": 1,
     "partialResult": false,
     "exceptions": [],
     "numGroupsLimitReached": false,
     "numGroupsWarningLimitReached": false,
     "timeUsedMs": 10,
     "controllerExecutionTimeMs": 25,
     "requestId": "1648150687000000002",
     "clientRequestId": null,
     "brokerId": "Broker_localhost_8099",
     "numDocsScanned": 97889,
     "totalDocs": 97889,
     "numEntriesScannedInFilter": 0,
     "numEntriesScannedPostFilter": 0,
     "numServersQueried": 1,
     "numServersResponded": 1,
     "numSegmentsQueried": 1,
     "numSegmentsProcessed": 1,
     "numSegmentsMatched": 1,
     "numConsumingSegmentsQueried": 0,
     "numConsumingSegmentsProcessed": 0,
     "numConsumingSegmentsMatched": 0,
     "minConsumingFreshnessTimeMs": 0,
     "numSegmentsPrunedByBroker": 0,
     "numSegmentsPrunedByServer": 0,
     "numSegmentsPrunedInvalid": 0,
     "numSegmentsPrunedByLimit": 0,
     "numSegmentsPrunedByValue": 0,
     "brokerReduceTimeMs": 0,
     "offlineThreadCpuTimeNs": 0,
     "realtimeThreadCpuTimeNs": 0,
     "offlineSystemActivitiesCpuTimeNs": 0,
     "realtimeSystemActivitiesCpuTimeNs": 0,
     "offlineResponseSerializationCpuTimeNs": 0,
     "realtimeResponseSerializationCpuTimeNs": 0,
     "offlineTotalCpuTimeNs": 0,
     "realtimeTotalCpuTimeNs": 0,
     "explainPlanNumEmptyFilterSegments": 0,
     "explainPlanNumMatchAllFilterSegments": 0,
     "traceInfo": {},
     "tablesQueried": [
       "baseballStats"
     ],
     "offlineThreadMemAllocatedBytes": 0,
     "realtimeThreadMemAllocatedBytes": 0,
     "offlineResponseSerMemAllocatedBytes": 0,
     "realtimeResponseSerMemAllocatedBytes": 0,
     "offlineTotalMemAllocatedBytes": 0,
     "realtimeTotalMemAllocatedBytes": 0,
     "pools": [
       -1
     ],
     "rlsFiltersApplied": false,
     "groupsTrimmed": false
   }


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "Akanksha-kedia (via GitHub)" <gi...@apache.org>.
Akanksha-kedia closed pull request #17647: Add controller execution time tracking to query responses
URL: https://github.com/apache/pinot/pull/17647


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "Akanksha-kedia (via GitHub)" <gi...@apache.org>.
Akanksha-kedia commented on PR #17647:
URL: https://github.com/apache/pinot/pull/17647#issuecomment-3902232963

   curl -X POST http://localhost:8099/query/sql -H "Content-Type: application/json" -d '{"sql":"SELECT COUNT(*) FROM baseballStats"}' | jq .
     % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                    Dload  Upload   Total   Spent    Left  Speed
   100  1610  100  1566  100    44  22778    640 --:--:-- --:--:-- --:--:-- 23676
   {
     "resultTable": {
       "dataSchema": {
         "columnNames": [
           "count(*)"
         ],
         "columnDataTypes": [
           "LONG"
         ]
       },
       "rows": [
         [
           97889
         ]
       ]
     },
     "numRowsResultSet": 1,
     "partialResult": false,
     "exceptions": [],
     "numGroupsLimitReached": false,
     "numGroupsWarningLimitReached": false,
     "timeUsedMs": 37,
     "controllerExecutionTimeMs": -1,
     "requestId": "1648150687000000004",
     "clientRequestId": null,
     "brokerId": "Broker_localhost_8099",
     "numDocsScanned": 97889,
     "totalDocs": 97889,
     "numEntriesScannedInFilter": 0,
     "numEntriesScannedPostFilter": 0,
     "numServersQueried": 1,
     "numServersResponded": 1,
     "numSegmentsQueried": 1,
     "numSegmentsProcessed": 1,
     "numSegmentsMatched": 1,
     "numConsumingSegmentsQueried": 0,
     "numConsumingSegmentsProcessed": 0,
     "numConsumingSegmentsMatched": 0,
     "minConsumingFreshnessTimeMs": 0,
     "numSegmentsPrunedByBroker": 0,
     "numSegmentsPrunedByServer": 0,
     "numSegmentsPrunedInvalid": 0,
     "numSegmentsPrunedByLimit": 0,
     "numSegmentsPrunedByValue": 0,
     "brokerReduceTimeMs": 0,
     "offlineThreadCpuTimeNs": 0,
     "realtimeThreadCpuTimeNs": 0,
     "offlineSystemActivitiesCpuTimeNs": 0,
     "realtimeSystemActivitiesCpuTimeNs": 0,
     "offlineResponseSerializationCpuTimeNs": 0,
     "realtimeResponseSerializationCpuTimeNs": 0,
     "offlineTotalCpuTimeNs": 0,
     "realtimeTotalCpuTimeNs": 0,
     "explainPlanNumEmptyFilterSegments": 0,
     "explainPlanNumMatchAllFilterSegments": 0,
     "traceInfo": {},
     "tablesQueried": [
       "baseballStats"
     ],
     "offlineThreadMemAllocatedBytes": 0,
     "realtimeThreadMemAllocatedBytes": 0,
     "offlineResponseSerMemAllocatedBytes": 0,
     "realtimeResponseSerMemAllocatedBytes": 0,
     "offlineTotalMemAllocatedBytes": 0,
     "realtimeTotalMemAllocatedBytes": 0,
     "pools": [
       -1
     ],
     "rlsFiltersApplied": false,
     "groupsTrimmed": false
   }
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "Akanksha-kedia (via GitHub)" <gi...@apache.org>.
Akanksha-kedia commented on PR #17647:
URL: https://github.com/apache/pinot/pull/17647#issuecomment-3902233526

   @shauryachats please see
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17647:
URL: https://github.com/apache/pinot/pull/17647#issuecomment-3858304360

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17647?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 34.05%. Comparing base ([`ca952da`](https://app.codecov.io/gh/apache/pinot/commit/ca952da64b1e282b3cb20e269bd3bf1af4d5491b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4ea51d9`](https://app.codecov.io/gh/apache/pinot/commit/4ea51d972ea3fbb80f429265b424e41730bafe72?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 8 commits behind head on master.
   :white_check_mark: All tests successful. No failed tests found.
   > :exclamation:  There is a different number of reports uploaded between BASE (ca952da) and HEAD (4ea51d9). Click for more details.
   > 
   > <details><summary>HEAD has 32 uploads less than BASE</summary>
   >
   >| Flag | BASE (ca952da) | HEAD (4ea51d9) |
   >|------|------|------|
   >|java-21|4|1|
   >|unittests1|2|0|
   >|unittests|4|1|
   >|temurin|9|1|
   >|java-11|5|0|
   >|unittests2|2|1|
   >|integration|5|0|
   >|integration1|2|0|
   >|custom-integration1|2|0|
   >|integration2|1|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #17647       +/-   ##
   =============================================
   - Coverage     63.20%   34.05%   -29.16%     
   + Complexity     1499      773      -726     
   =============================================
     Files          3173     3174        +1     
     Lines        190257   190321       +64     
     Branches      29072    29080        +8     
   =============================================
   - Hits         120261    64808    -55453     
   - Misses        60662   120147    +59485     
   + Partials       9334     5366     -3968     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-21](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.05% <100.00%> (-29.11%)` | :arrow_down: |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.05% <100.00%> (-29.16%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.05% <100.00%> (-29.16%)` | :arrow_down: |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17647/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.05% <100.00%> (+0.02%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17647?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add controller execution time tracking to query responses [pinot]

Posted by "shauryachats (via GitHub)" <gi...@apache.org>.
shauryachats commented on PR #17647:
URL: https://github.com/apache/pinot/pull/17647#issuecomment-3893555961

   @Akanksha-kedia is this only for queries executed through Controller UI? If yes, could we instead consider modifying only the controller APIs to additionally bundle the controller execution time given that most of the queries would generally be through the broker itself and this increases the response size without providing any additional information for broker queries.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org