You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2026/08/07 21:50:27 UTC

[PR] [CI] Balance integration tests across three shards [pinot]

xiangfu0 opened a new pull request, #19190:
URL: https://github.com/apache/pinot/pull/19190

   ## What changed
   
   - Remove `LLCRealtimeKafka3ClusterIntegrationTest`, which duplicated the generic LLC realtime suite after that suite migrated to Kafka 3.
   - Split the main integration tests into three duration-balanced, prefix-based shards.
   - Keep the custom shared-cluster suite on shard 1 and preserve the existing per-shard heap limits without overriding JaCoCo's agent arguments.
   
   ## Why
   
   The [measured integration-test job](https://github.com/apache/pinot/actions/runs/31212786223/job/92979285734?pr=19181) spent 58:48 in the main serial suite and another 8:07 in the custom suite. The redundant Kafka 3 class alone took about 4:35 while running the same 19 inherited tests as `LLCRealtimeClusterIntegrationTest`.
   
   Using the observed class durations after removing that duplicate, the three shard workloads are balanced at approximately 40:21, 40:24, and 40:12. The projected critical path is 50-52 minutes instead of 75:29, a reduction of roughly 25 minutes (33%). This PR's CI run will validate that projection.
   
   ## Validation
   
   - `LLCRealtimeClusterIntegrationTest`: 19 tests, 0 failures, 0 errors, 2 skipped
   - Maven reactor: success
   - `spotless:apply`, `license:format`, `checkstyle:check`, and `license:check`
   - Workflow YAML, shell, and POM XML syntax checks
   - Verified every A-Z prefix occurs in exactly one shard
   - Verified JaCoCo prepends its agent to the 2 GiB shard JVM arguments
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] [CI] Balance integration tests across three shards [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19190:
URL: https://github.com/apache/pinot/pull/19190#issuecomment-5222860106

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/19190?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 100.00%. Comparing base ([`690ac21`](https://app.codecov.io/gh/apache/pinot/commit/690ac21ffb6c90486e9994ca9f3b32608057e30a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`7501674`](https://app.codecov.io/gh/apache/pinot/commit/7501674064e845d00b244b527bb1aaa44d845e08?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff               @@
   ##             master    #19190       +/-   ##
   ==============================================
   + Coverage     66.60%   100.00%   +33.39%     
   + Complexity     1423         6     -1417     
   ==============================================
     Files          3443         3     -3440     
     Lines        218633         6   -218627     
     Branches      34795         0    -34795     
   ==============================================
   - Hits         145620         6   -145614     
   + Misses        61283         0    -61283     
   + Partials      11730         0    -11730     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (ø)` | |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (ø)` | |
   | [java-25](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+33.39%)` | :arrow_up: |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ø> (+33.39%)` | :arrow_up: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/19190/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/pinot/pull/19190?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] [CI] Consolidate ingestion integration test fixtures [pinot]

Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #19190:
URL: https://github.com/apache/pinot/pull/19190


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org