You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "RocMarshal (via GitHub)" <gi...@apache.org> on 2024/11/12 14:40:04 UTC
[PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
RocMarshal opened a new pull request, #25647:
URL: https://github.com/apache/flink/pull/25647
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
- [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum
- [FLINK-36691][runtime] Support configuration to enable tasks balancing at slot level for Default Scheduler
## Brief change log
- [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum
- [FLINK-36691][runtime] Support configuration to enable tasks balancing at slot level for Default Scheduler
## Verifying this change
This change is already covered by existing tests,
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (**yes** / no / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (**yes** / no)
- If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2478033320
Thank @zhuzhurk for the reply.
> The feature freeze date has been postponed to 2025-01-13[1]. Will it give enough time to complete the FLIP?
Currently we only have one core ticket FLINK-33388 which needs some additional development.
The rest of the work will be focused on the revision of the core ticket.
Therefore, I think it will be no problem to complete this FLIP before the deadline.
CC @1996fanrui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2473018279
Thanks @zhuzhurk @1996fanrui for the comments.
Please let me to synchronize the background information and introduce the reasons for this change.
The feature freeze for version 2.0 will take place at the end of November 2024.
> Does you mean that the balance of Slot to TM[1] is not planned to be supported in Flink 2.0?
We expect to release this complete feature in 2.0 ideally
but I estimate that the two core PRs at the TaskManager level may face some risks based on the current progress,
- https://github.com/apache/flink/pull/25517(Support TM level balancing for AdaptiveSheduler)
- https://github.com/apache/flink/pull/25174(Support TM level balancing for DefaultSheduler)
Meanwhile, for FLIP-370,
- enabling the switch to support slot-level balancing could address a significant portion of the tasks skew issues.
- The solutions for these two PRs are already determined and are currently awaiting review.
- https://github.com/apache/flink/pull/25647
- https://github.com/apache/flink/pull/25504
Therefore, I'd like to attempt to first support the slot-level balancing for the two schedulers.
Please let me know what's your opinion~ Thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2490489348
Reviewed by Chi on 21/11/24. Asked submitter questions. Mostly to make it clear when to configure this new option, by bringing in appropriate Flip content and reasoning into the docs. Ideally including diagrams to show the need for the new option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2472350285
hi, @zhuzhurk @1996fanrui Could you help take a look for the PR if you had the free time ?
Many thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "zhuzhurk (via GitHub)" <gi...@apache.org>.
zhuzhurk commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2477797783
> Thanks @zhuzhurk @1996fanrui for the comments.
>
> Please let me to synchronize the background information and introduce the reasons for this change.
>
> The feature freeze for version 2.0 will take place at the end of November 2024.
>
> > Does you mean that the balance of Slot to TM[1] is not planned to be supported in Flink 2.0?
>
> We expect to release this complete feature in 2.0 ideally but I estimate that the two core PRs at the TaskManager level may face some risks based on the current progress,
>
> * [[FLINK-33391][runtime] Support tasks balancing at TM level for Adaptive Scheduler. #25517](https://github.com/apache/flink/pull/25517) TM level balancing for AdaptiveSheduler)
> * [[FLINK-33387][runtime] Introduce the abstraction and the interface about loading #25174](https://github.com/apache/flink/pull/25174) TM level balancing for DefaultSheduler)
>
> Meanwhile, for FLIP-370,
>
> * enabling the switch to support slot-level balancing could address a significant portion of the tasks skew issues.
> * The solutions for these two PRs are already determined and are currently awaiting review.
>
> * [[FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum #25647](https://github.com/apache/flink/pull/25647)
> * [[FLINK-33389][runtime] Support tasks balancing at slot level for Adaptive Scheduler #25504](https://github.com/apache/flink/pull/25504)
>
> Therefore, I'd like to attempt to first support the slot-level balancing for the two schedulers.
>
> Please let me know what's your opinion~ Thx!
> The feature freeze for version 2.0 will take place at the end of November 2024
The feature freeze date has been postponed to 2025-01-13[1]. Will it give enough time to complete the FLIP?
[1] [DISCUSS] Feature Freeze for Flink 2.0 - Jan 13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal closed pull request #25647: [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum
URL: https://github.com/apache/flink/pull/25647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2470727965
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9021781fb347becca7825f9288e8fefcd2c5a8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1c9021781fb347becca7825f9288e8fefcd2c5a8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9021781fb347becca7825f9288e8fefcd2c5a8 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1852202449
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/SlotSharingExecutionSlotAllocatorFactory.java:
##########
@@ -40,13 +41,16 @@ public SlotSharingExecutionSlotAllocatorFactory(
PhysicalSlotProvider slotProvider,
boolean slotWillBeOccupiedIndefinitely,
PhysicalSlotRequestBulkChecker bulkChecker,
- Duration allocationTimeout) {
+ Duration allocationTimeout,
+ TaskManagerOptions.TaskManagerLoadBalanceMode taskManagerLoadBalanceMode) {
this(
slotProvider,
slotWillBeOccupiedIndefinitely,
bulkChecker,
allocationTimeout,
- new LocalInputPreferredSlotSharingStrategy.Factory());
+ taskManagerLoadBalanceMode == TaskManagerOptions.TaskManagerLoadBalanceMode.TASKS
+ ? new TaskBalancedPreferredSlotSharingStrategy.Factory()
Review Comment:
In fact, the phenomenon of uneven task scheduling results is mainly caused by two stages:
1. tasks->slot
2. slots->taskmanager
Because prior to the initial feature freeze, we anticipated that the entire FLIP would be at risk. Our original intention in initiating this PR was to release this change as early as possible to address the imbalance issue in the task ->slot dimension. But now there is no time risk with this feature, so we plan to release both dimensions of functionality together.
BTW, the optimizations and changes mentioned above are highly likely not to be fixed in the current PR as it may be abandoned.
But we will place the corresponding optimization in the appropriate PR position in the future. If you are willing to help with the review, I would greatly appreciate it! :)
##########
docs/layouts/shortcodes/generated/all_taskmanager_section.html:
##########
@@ -90,7 +90,7 @@
<td><h5>taskmanager.load-balance.mode</h5></td>
<td style="word-wrap: break-word;">NONE</td>
<td><p>Enum</p></td>
- <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li></ul></td>
+ <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">TASKS</code> mode tries to schedule evenly all tasks based on its' number across all available <code class="highlighter-rouge">TaskManagers</code>. Note: Currently, enabling this parameter can only achieve the balancing effect of the slot level dimension of <code class="highlighter-rouge">DefaultScheduler</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li><li>"TASKS"</li></ul></td>
Review Comment:
Good idea~, we do need a more detailed and precise description, as well as traceable design documents.
How about making the following changes based on your suggestion?
1. Optimize the configuration description section in the code, including adding FLIP addresses
2. The explanation of how to use and the differences between it and other configurations, as well as the creation of charts, can be placed in https://issues.apache.org/jira/browse/FLINK-33392 Completed in the middle
##########
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java:
##########
@@ -707,6 +707,12 @@ public class TaskManagerOptions {
"The %s mode tries to spread out the slots evenly across all available %s.",
code(TaskManagerLoadBalanceMode.SLOTS.name()),
code("TaskManagers")),
+ text(
+ "The %s mode tries to schedule evenly all tasks based on its' number across all available %s. "
Review Comment:
The meaning expressed here is that the scheduler will try its best to ensure that the scheduling results are concentrated and the number of tasks on each Taskmanager is similar.
I will add a more detailed description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851640555
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/SlotSharingExecutionSlotAllocatorFactory.java:
##########
@@ -40,13 +41,16 @@ public SlotSharingExecutionSlotAllocatorFactory(
PhysicalSlotProvider slotProvider,
boolean slotWillBeOccupiedIndefinitely,
PhysicalSlotRequestBulkChecker bulkChecker,
- Duration allocationTimeout) {
+ Duration allocationTimeout,
+ TaskManagerOptions.TaskManagerLoadBalanceMode taskManagerLoadBalanceMode) {
this(
slotProvider,
slotWillBeOccupiedIndefinitely,
bulkChecker,
allocationTimeout,
- new LocalInputPreferredSlotSharingStrategy.Factory());
+ taskManagerLoadBalanceMode == TaskManagerOptions.TaskManagerLoadBalanceMode.TASKS
+ ? new TaskBalancedPreferredSlotSharingStrategy.Factory()
Review Comment:
I see that specifying TASKS means this factory will be used to load balance. I do not see the works task number in the factory implementation. It seems to be trying to colocate tasks in a slot. I am not sure when this would be preferable or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "zhuzhurk (via GitHub)" <gi...@apache.org>.
zhuzhurk commented on PR #25647:
URL: https://github.com/apache/flink/pull/25647#issuecomment-2472879330
> As I understand, task balancing needs 2 level:
>
> * Task to Slot
> * Slot to TM
>
> This PR will ` Introduce the TASKS for TaskManagerLoadBalanceMode enum`, and only support task balancing at slot level. And `Slot to TM` will be supported later.
>
> I think it's acceptable for Flink 2.0, and I hope hear the suggestion from @zhuzhurk .
Does you mean that the balance of `Slot to TM`[1] is not planned to be supported in Flink 2.0?
[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-370%3A+Support+Balanced+Tasks+Scheduling#FLIP370:SupportBalancedTasksScheduling-3.2.3ImplementslotstoTMsbalancingforDefaultScheduler:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851640826
##########
docs/layouts/shortcodes/generated/all_taskmanager_section.html:
##########
@@ -90,7 +90,7 @@
<td><h5>taskmanager.load-balance.mode</h5></td>
<td style="word-wrap: break-word;">NONE</td>
<td><p>Enum</p></td>
- <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li></ul></td>
+ <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">TASKS</code> mode tries to schedule evenly all tasks based on its' number across all available <code class="highlighter-rouge">TaskManagers</code>. Note: Currently, enabling this parameter can only achieve the balancing effect of the slot level dimension of <code class="highlighter-rouge">DefaultScheduler</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li><li>"TASKS"</li></ul></td>
Review Comment:
I am not sure what this sentence means - it would be really useful to have a diagram, and maybe an example showing the SLOT and TASK modes and how they differ.
It would also be useful to detail why you would choose TASK mode of SLOT mode.
I was looking for more information in the Jira - it should point to the Flip .
I see some of the conversations in the PR as to why this might be needed. Could we summarize these considerations in the docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851640826
##########
docs/layouts/shortcodes/generated/all_taskmanager_section.html:
##########
@@ -90,7 +90,7 @@
<td><h5>taskmanager.load-balance.mode</h5></td>
<td style="word-wrap: break-word;">NONE</td>
<td><p>Enum</p></td>
- <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li></ul></td>
+ <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">TASKS</code> mode tries to schedule evenly all tasks based on its' number across all available <code class="highlighter-rouge">TaskManagers</code>. Note: Currently, enabling this parameter can only achieve the balancing effect of the slot level dimension of <code class="highlighter-rouge">DefaultScheduler</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li><li>"TASKS"</li></ul></td>
Review Comment:
I am not sure what this sentence means - it would be really useful to have a diagram, and maybe an example showing the SLOT and TASK modes and how they differ.
It would also be useful to detail why you would choose TASK mode of SLOT mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851640826
##########
docs/layouts/shortcodes/generated/all_taskmanager_section.html:
##########
@@ -90,7 +90,7 @@
<td><h5>taskmanager.load-balance.mode</h5></td>
<td style="word-wrap: break-word;">NONE</td>
<td><p>Enum</p></td>
- <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li></ul></td>
+ <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">TASKS</code> mode tries to schedule evenly all tasks based on its' number across all available <code class="highlighter-rouge">TaskManagers</code>. Note: Currently, enabling this parameter can only achieve the balancing effect of the slot level dimension of <code class="highlighter-rouge">DefaultScheduler</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li><li>"TASKS"</li></ul></td>
Review Comment:
I am not sure what this sentence means - it would be really useful to have a diagram, and maybe an example showing the SLOT and TASK modes and how they differ.
It would also be useful to detail why you would choose TASK mode of SLOT mode.
There is no information in the Jira explaining this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851618895
##########
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java:
##########
@@ -707,6 +707,12 @@ public class TaskManagerOptions {
"The %s mode tries to spread out the slots evenly across all available %s.",
code(TaskManagerLoadBalanceMode.SLOTS.name()),
code("TaskManagers")),
+ text(
Review Comment:
NOT: its' -> it's
##########
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java:
##########
@@ -707,6 +707,12 @@ public class TaskManagerOptions {
"The %s mode tries to spread out the slots evenly across all available %s.",
code(TaskManagerLoadBalanceMode.SLOTS.name()),
code("TaskManagers")),
+ text(
Review Comment:
NIT: its' -> it's
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35966][runtime] Introduce the TASKS for TaskManagerLoadBalanceMode enum [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #25647:
URL: https://github.com/apache/flink/pull/25647#discussion_r1851620823
##########
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java:
##########
@@ -707,6 +707,12 @@ public class TaskManagerOptions {
"The %s mode tries to spread out the slots evenly across all available %s.",
code(TaskManagerLoadBalanceMode.SLOTS.name()),
code("TaskManagers")),
+ text(
+ "The %s mode tries to schedule evenly all tasks based on its' number across all available %s. "
Review Comment:
I don't understand this sentence. I am not sure what "schedule evenly all tasks based on its' number" means.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org