You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "davedwwang (via GitHub)" <gi...@apache.org> on 2026/02/14 11:52:45 UTC
[PR] [AMORO-4086] Add OptimizerGroupKeeper to automatically maintain optim… [amoro]
davedwwang opened a new pull request, #4087:
URL: https://github.com/apache/amoro/pull/4087
## Why are the changes needed?
Currently, when users configure optimizer resources through the dashboard, there is no automatic mechanism to ensure the resource group always meets the minimum parallelism requirement. Users must manually monitor and create optimizers when resources fall below the expected level, which is inconvenient for production environments where automatic resource management is expected.
Close #4086.
## Brief change log
- OptimizerGroupKeepingTask - Delayed task for tracking resource group keeping status
- OptimizerGroupKeeper - Background thread extending AbstractKeeper for monitoring and maintaining optimizer resources
- AbstractKeeper<T> - Refactored common base class for both OptimizerKeeper and OptimizerGroupKeeper
## How was this patch tested?
- [ x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x ] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4086] Add OptimizerGroupKeeper to automatically maintain optim… [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on PR #4087:
URL: https://github.com/apache/amoro/pull/4087#issuecomment-4023410258
The new `min-parallelism` property should be documented in the optimizer group properties table at `docs/admin-guides/managing-optimizers.md`, under the `## Optimizer group` section (around the existing properties table in `### Add optimizer group`).
Suggested addition to the properties table:
| Property | Container type | Required | Default | Description |
|---|---|---|---|---|
| min-parallelism | All | No | 0 | The minimum total parallelism (CPU cores) that the optimizer group should maintain. When the current total cores of all running optimizers fall below this value, `OptimizerGroupKeeper` will automatically scale out new optimizers. Set to `0` to disable auto-scaling. |
Also worth noting that the AMS-level configs `optimizer-group.min-parallelism-check-interval` and `optimizer-group.max-keeping-attempts` (already added to `docs/configuration/ams-config.md`) control the behavior of this auto-scaling mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4086] Add OptimizerGroupKeeper to automatically maintain optim… [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong merged PR #4087:
URL: https://github.com/apache/amoro/pull/4087
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org