You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "shounakmk219 (via GitHub)" <gi...@apache.org> on 2026/01/06 07:19:10 UTC
[PR] Refactor PinotTaskManager to dedup code and improve readability [pinot]
shounakmk219 opened a new pull request, #17459:
URL: https://github.com/apache/pinot/pull/17459
## Summary
This PR refactors `PinotTaskManager` to eliminate code duplication between ad-hoc and scheduled task creation flows, and improves overall code readability and maintainability.
## Changes
Extracted helper methods to reduce duplication between `createTask()` (ad-hoc) and `scheduleTask()` (periodic) flows:
- `prepTaskQueue()` - Centralizes task queue preparation and metrics initialization
- `getParentTaskName()` - Handles task name generation, validation, and duplicate checking
- `getTableNameWithTypes()` - Extracts and validates table names with types
- `getTaskGenerator()` - Retrieves and validates task generator with proper error handling
- `isExceedingResourceUtilizationLimits()` - Centralizes resource utilization checking logic
- `acquireTaskLock()` - Unified distributed lock acquisition for both ad-hoc and scheduled tasks
- `validatePinotTaskConfigs()` - Validates task configs and enforces maximum subtask limits
- `submitTasks()` - Unified task submission logic for both flows
- `getTableNames()` - Helper to extract table names from table configs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Refactor PinotTaskManager to dedup code and improve readability [pinot]
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish merged PR #17459:
URL: https://github.com/apache/pinot/pull/17459
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Refactor PinotTaskManager to dedup code and improve readability [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17459:
URL: https://github.com/apache/pinot/pull/17459#issuecomment-3713597813
### :x: 6 Tests Failed:
| Tests completed | Failed | Passed | Skipped |
|---|---|---|---|
| 6004 | 6 | 5998 | 16 |
<details><summary>View the top 2 failed test(s) by shortest run time</summary>
> <pre><code class="language-python">org.apache.pinot.controller.helix.HelixHelperTest::@BeforeClass setUp</code></pre>
> <details><summary>Stack Traces | 0.018s run time</summary>
>
> > <pre><code class="language-python">expected [true] but found [false]</code></pre>
>
> </details>
> <pre><code class="language-python">org.apache.pinot.controller.helix.HelixHelperTest::setUp</code></pre>
> <details><summary>Stack Traces | 0.018s run time</summary>
>
> > <pre><code class="language-python">expected [true] but found [false]</code></pre>
>
> </details>
</details>
<details><summary>View the full list of 4 :snowflake: flaky test(s)</summary>
> <pre><code class="language-python">org.apache.pinot.controller.helix.LogicalTableMetadataCacheTest::@BeforeMethod beforeMethod</code></pre>
> **Flake rate in main:** 100.00% (Passed 0 times, Failed 5 times)
> <details><summary>Stack Traces | 0.016s run time</summary>
>
> > <pre><code class="language-python">org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 409 (Conflict) with reason: &quot;External view for testTable_OFFLINE still exists. If the table is just deleted, please wait for the clean up to finish before recreating it. If the external view is not removed after a long time, try restarting the servers showing up in the external view&quot; while sending request: /tables to controller: runnervmh13bl.mhuo5aufw0zudj5rd413yb3ebd.dx.internal.cloudapp.net, version: Unknown</code></pre>
>
> </details>
> <pre><code class="language-python">org.apache.pinot.controller.helix.LogicalTableMetadataCacheTest::beforeMethod</code></pre>
> **Flake rate in main:** 100.00% (Passed 0 times, Failed 5 times)
> <details><summary>Stack Traces | 0.118s run time</summary>
>
> > <pre><code class="language-python">org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 409 (Conflict) with reason: "External view for testTable_OFFLINE still exists. If the table is just deleted, please wait for the clean up to finish before recreating it. If the external view is not removed after a long time, try restarting the servers showing up in the external view" while sending request: /tables to controller: runnervmh13bl.mhuo5aufw0zudj5rd413yb3ebd.dx.internal.cloudapp.net, version: Unknown</code></pre>
>
> </details>
> <pre><code class="language-python">org.apache.pinot.controller.validation.ValidationManagerTest::@BeforeClass setUp</code></pre>
> **Flake rate in main:** 100.00% (Passed 0 times, Failed 5 times)
> <details><summary>Stack Traces | 0.006s run time</summary>
>
> > <pre><code class="language-python">expected [true] but found [false]</code></pre>
>
> </details>
> <pre><code class="language-python">org.apache.pinot.controller.validation.ValidationManagerTest::setUp</code></pre>
> **Flake rate in main:** 100.00% (Passed 0 times, Failed 5 times)
> <details><summary>Stack Traces | 0.006s run time</summary>
>
> > <pre><code class="language-python">expected [true] but found [false]</code></pre>
>
> </details>
</details>
To view more test analytics, go to the [Test Analytics Dashboard](https://app.codecov.io/gh/apache/pinot/tests/shounakmk219%3Atask-manager-refactor)
<sub>📋 Got 3 mins? [Take this short survey](https://forms.gle/22i53Qa1CySZjA6c7) to help us improve Test Analytics.</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org