You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "ruanwenjun (via GitHub)" <gi...@apache.org> on 2026/01/19 14:39:17 UTC
[I] [Improvement][API] Use worker group id rather than worker group name in EnvironmentWorkerGroupRelation [dolphinscheduler]
ruanwenjun opened a new issue, #17890:
URL: https://github.com/apache/dolphinscheduler/issues/17890
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Improvement][API] The workergroup used at EnvironmentWorkerGroupRelation/TaskDefinition cannot work when workergroup name changed [dolphinscheduler]
Posted by "GaneshPatil7517 (via GitHub)" <gi...@apache.org>.
GaneshPatil7517 commented on issue #17890:
URL: https://github.com/apache/dolphinscheduler/issues/17890#issuecomment-3852007216
hey @ruanwenjun if this issue is open please assign me i want to work on this........
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Improvement][API] The workergroup used at EnvironmentWorkerGroupRelation/TaskDefinition cannot work when workergroup name changed [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on issue #17890:
URL: https://github.com/apache/dolphinscheduler/issues/17890#issuecomment-3853095229
@GaneshPatil7517 Please provide some design, if you have any idea. I'm currently handling this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Improvement][API] The workergroup used at EnvironmentWorkerGroupRelation/TaskDefinition cannot work when workergroup name changed [dolphinscheduler]
Posted by "GaneshPatil7517 (via GitHub)" <gi...@apache.org>.
GaneshPatil7517 commented on issue #17890:
URL: https://github.com/apache/dolphinscheduler/issues/17890#issuecomment-3857743417
<html>
<body>
<!--StartFragment--><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>Hi @ruanwenjun,</strong></p><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-s
troke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Thanks for your response. Here's my design proposal for this issue:</p><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Problem</h3><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: n
ormal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">When a worker group's name is changed,<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#30%2C14" custom-hover="true" data-keybinding-context="2485" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; b
order-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: rgb(204, 204, 204); color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">EnvironmentWorkerGroupRelation</span></a><span> </span>and<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java#56%2C14" custom-hover="true" data-keybinding-context="2486" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px
; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: rgb(204, 204, 204); color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">TaskDefinition</span></a><span> </span>become orphaned because they reference the worker group by name instead of ID.</p><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Proposed Solution</h3><p style="margin: 0px 0px 16px; color:
rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>1. Database Schema Changes</strong></p>
Table | Change
-- | --
t_ds_environment_worker_group_relation | Add worker_group_id INT column (nullable initially for backward compatibility)
t_ds_task_definition | Add worker_group_id INT column (nullable initially for backward compatibility)
t_ds_task_definition_log | Add worker_group_id INT column
<p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>2. Entity Changes</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width
: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Add<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">workerGroupId</code><span> </span>field to<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#30%2C14" custom-hover="true" data-keybinding-context="2
487" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">EnvironmentWorkerGroupRelation</span></a>,<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java#56%2C14" custom-hover="true" data-keybinding-context="2488" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-s
tyle: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">TaskDefinition</span></a>, and<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">TaskDefinitionLog</code></li><li>Keep<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinschedule
r/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#43%2C20" custom-hover="true" data-keybinding-context="2489" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">workerGroup</span></a><span> </span>(name) field for backward compatibility during transition</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text
-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>3. API Changes</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Modify create/update APIs to accept<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999
px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">workerGroupId</code><span> </span>instead of (or in addition to)<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#43%2C20" custom-hover="true" data-keybinding-context="2490" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: no
wrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">workerGroup</span></a><span> </span>name</li><li>Add validation to ensure referenced worker group exists</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>4. Migration Strategy</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Sego
e UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Provide database migration script to populate<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">worker_group_id</code><span> </span>based on existing<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size
: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">worker_group</code><span> </span>name</li><li>Deprecate name-based references in future versions</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>5. Worker Registration</strong><br>For dynamically registered workers (#179
29):</p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Require worker groups to be pre-created via UI before workers can join</li><li>Workers attempting to join non-existent groups should fail with clear error message</li></ul><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-varia
nt-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Implementation Order</h3><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Following the sub-issues:</p><ol style="padding-inline-start: 28px; color: rgb(204,
204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>#17926 - Create worker groups without workers (prerequisite)</li><li>#17928 - Add workerGroupId to EnvironmentWorkerGroupRelation</li><li>#17930 - Add workerGroupId to TaskDefinition</li><li>#17929 - Prevent workers joining non-existent groups</li></ol><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-w
eight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">I'd be happy to implement any of these sub-issues. Let me know which one you'd like me to start with!</p><hr style="border-color: rgba(255, 255, 255, 0.18); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><br class="Apple
-interchange-newline"><!--EndFragment-->
</body>
</html>Hi @ruanwenjun,
Thanks for your response. Here's my design proposal for this issue:
Problem
When a worker group's name is changed, [EnvironmentWorkerGroupRelation](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) and [TaskDefinition](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) become orphaned because they reference the worker group by name instead of ID.
Proposed Solution
1. Database Schema Changes
Table Change
t_ds_environment_worker_group_relation Add worker_group_id INT column (nullable initially for backward compatibility)
t_ds_task_definition Add worker_group_id INT column (nullable initially for backward compatibility)
t_ds_task_definition_log Add worker_group_id INT column
2. Entity Changes
Add workerGroupId field to [EnvironmentWorkerGroupRelation](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html), [TaskDefinition](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html), and TaskDefinitionLog
Keep [workerGroup](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) (name) field for backward compatibility during transition
3. API Changes
Modify create/update APIs to accept workerGroupId instead of (or in addition to) [workerGroup](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) name
Add validation to ensure referenced worker group exists
4. Migration Strategy
Provide database migration script to populate worker_group_id based on existing worker_group name
Deprecate name-based references in future versions
5. Worker Registration
For dynamically registered workers (#17929):
Require worker groups to be pre-created via UI before workers can join
Workers attempting to join non-existent groups should fail with clear error message
Implementation Order
Following the sub-issues:
#17926 - Create worker groups without workers (prerequisite)
#17928 - Add workerGroupId to EnvironmentWorkerGroupRelation
#17930 - Add workerGroupId to TaskDefinition
#17929 - Prevent workers joining non-existent groups
I'd be happy to implement any of these sub-issues. Let me know which one you'd like me to start with!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Improvement][API] The EnvironmentWorkerGroupRelation cannot use when workergroup name change [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on issue #17890:
URL: https://github.com/apache/dolphinscheduler/issues/17890#issuecomment-3830349680
In DolphinScheduler, there are two types of worker groups: one defined through the UI, and the other defined in worker configurations and registered dynamically at runtime.
The current issue is that worker groups registered dynamically by workers do not have an ID and lack associated metadata. As a result, we can only rely on the worker group name for identification and association.
For scenarios that require dynamic worker registration, we can still require worker groups to be created in advance through the UI. When a worker starts, it will then automatically register itself to the specified worker group. This approach ensures that all worker groups have consistent metadata while still supporting dynamic worker registration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org