You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "gortiz (via GitHub)" <gi...@apache.org> on 2025/06/02 16:57:44 UTC

[PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

gortiz opened a new pull request, #15967:
URL: https://github.com/apache/pinot/pull/15967

   We have detected that some large queries may end up consuming too much memory for too long in the stats cache used to return stats on timeout.
   
   A way to solve that is to remove the stats as soon as the opchain finishes successfully. This should keep the cache almost empty (given most of the time opchains don't fail) at the cost of losing the stats of fast stages when other stage timeouts.
   
   This is not the best solution but one that is very easy to implement. We will work in better long term solutions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #15967:
URL: https://github.com/apache/pinot/pull/15967


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on PR #15967:
URL: https://github.com/apache/pinot/pull/15967#issuecomment-2931649778

   @gortiz : how long running were the queries that caused this? (what was their configured timeout)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #15967:
URL: https://github.com/apache/pinot/pull/15967#issuecomment-2933627063

   > how long running were the queries that caused this? (what was their configured timeout)
   
   They were running for less than 1 second. But the issue is not on their execution time but the fact that we keep a reference to the multi stage operator. Some of these operators keep a reference to large maps (ie joins and aggregates), which is what somehow _leaks_ memory. This simple PR fixes that by cleaning up successful executions, but reduces precision on the stats and doesn't protect us against queries that fail.
   
   I'm planning to change the code so that it does not keep the reference to multi-stage operators once the opchain finishes. But I think we should also release these large maps once the operator is no longer going to use them. I've been thinking about other places where we reference operators longer than expected, and the only case I've found is the new physical planner, which can generate larger stages that contain two joins. That means that the right side table of one of the joins will be kept referenced even after the EOS was sent from that operator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Invalidate from cache the stats of opchains that finish successfully. [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15967:
URL: https://github.com/apache/pinot/pull/15967#issuecomment-2931740345

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15967?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Project coverage is 56.46%. Comparing base [(`1a476de`)](https://app.codecov.io/gh/apache/pinot/commit/1a476dea72ebf1c15e14ded28ae60c385f71b2c3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`ab0576c`)](https://app.codecov.io/gh/apache/pinot/commit/ab0576ce287f71af583d79a866b5f375435a43bf?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   > Report is 178 commits behind head on master.
   
   > :exclamation:  There is a different number of reports uploaded between BASE (1a476de) and HEAD (ab0576c). Click for more details.
   > 
   > <details><summary>HEAD has 68 uploads less than BASE</summary>
   >
   >| Flag | BASE (1a476de) | HEAD (ab0576c) |
   >|------|------|------|
   >|java-21|10|1|
   >|unittests|6|1|
   >|unittests1|3|1|
   >|skip-bytebuffers-true|4|0|
   >|temurin|15|1|
   >|unittests2|3|0|
   >|skip-bytebuffers-false|8|0|
   >|java-11|5|0|
   >|integration|9|0|
   >|integration2|3|0|
   >|integration1|3|0|
   >|custom-integration1|3|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #15967      +/-   ##
   ============================================
   - Coverage     62.90%   56.46%   -6.44%     
   + Complexity     1386      690     -696     
   ============================================
     Files          2867     2270     -597     
     Lines        163354   124030   -39324     
     Branches      24952    19763    -5189     
   ============================================
   - Hits         102755    70035   -32720     
   + Misses        52847    48205    -4642     
   + Partials       7752     5790    -1962     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-21](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.46% <100.00%> (-6.36%)` | :arrow_down: |
   | [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.46% <100.00%> (-6.44%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.46% <100.00%> (-6.44%)` | :arrow_down: |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.46% <100.00%> (+0.64%)` | :arrow_up: |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15967/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15967?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org