[jira] [Comment Edited] (FLINK-31679) [Flink][UI] Show data exchange type on web ui

["Weijie Guo (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-31679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707524#comment-17707524 ] 

Weijie Guo edited comment on FLINK-31679 at 4/1/23 9:53 AM:
------------------------------------------------------------

Thanks for the proposal, but I'm a bit curious about whether this feature is really helpful to the vast majority of users. The main reasons are as follows:
- For streaming job, all edges are always {{PIPELINED_BOUNDED}}.
- For batch job, in most cases all edges are {{BLOCKING}}. 
- Most users will not use pipelined shuffle in batch scenario, and this may have a potential scheduling deadlock problem. This is also part of the reason why we introduced hybrid shuffle.
- Keep this in mind: We plan to make hybrid shuffle the only shuffle implementation for batch jobs in the next few releases, and will also cover streaming scenarios in the future.

It seems that the only significance of showing the {{ResultPartitionType}} in the edge is that in batch job, pipelined shuffles are used, but the planner will make some of then become Blocking. But this will no longer be the case in the near future. In most other cases, this information is of little significance.


was (Author: weijie guo):
Thanks for the proposal, but I'm a bit curious about whether this feature is really helpful to the vast majority of users. The main reasons are as follows:
- For streaming job, all edges are always {{PIPELINED_BOUNDED}}.
- For batch job, in most cases all edges are {{BLOCKING}}. 
- Most users will not use pipelined shuffle in batch scenario, and this may have a potential scheduling deadlock problem. This is also part of the reason why we introduced hybrid shuffle.
- Keep this in mind: We plan to make hybrid shuffle the only shuffle implementation for batch jobs in the next few releases, and will also cover streaming scenarios in the future.

It seems that the only significance of showing the {{ResultPartitionType}} in the edge is that in batch job, pipelined shuffles are used, but the planner will make some of then become Blocking. In most other cases, this information is of little significance.

> [Flink][UI] Show data exchange type on web ui
> ---------------------------------------------
>
>                 Key: FLINK-31679
>                 URL: https://issues.apache.org/jira/browse/FLINK-31679
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Web Frontend
>    Affects Versions: 1.18.0
>            Reporter: Weihua Hu
>            Priority: Major
>
> Flink supports multiple data exchange types (ResultPartitionType), which subject to multiple parameters, such as whether job is streaming or batch.
> I think display the data exchange types on the UI would be helpful.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)