You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "lining (Jira)" <ji...@apache.org> on 2019/11/19 09:40:00 UTC
[jira] [Comment Edited] (FLINK-14815) Expose network pool usage in
IOMetricsInfo
[ https://issues.apache.org/jira/browse/FLINK-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977303#comment-16977303 ]
lining edited comment on FLINK-14815 at 11/19/19 9:39 AM:
----------------------------------------------------------
For the pool usages, in the case of data skew, although the average is very low, the status of a task is not good. On the vertex, the pool usage could help users to locate the bottleneck point based on these values when there is no backpressure. If it is only displayed in the subtask, the user needs to view subtasks of each vertex. As you said, if one single task is backpressure, the vertex is backpressure, so maybe max is more important than the other two.
was (Author: lining):
For the pool usages, in the case of data skew, although the average is very low, the status of a task is not good. On the vertex, the pool usage could help users to locate the bottleneck point based on these values when there is no backpressure. If it is only displayed in the subtask, the user needs to view subtasks of each vertex.
> Expose network pool usage in IOMetricsInfo
> ------------------------------------------
>
> Key: FLINK-14815
> URL: https://issues.apache.org/jira/browse/FLINK-14815
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Network, Runtime / REST
> Reporter: lining
> Assignee: lining
> Priority: Major
>
> * If sub task is not back pressured, but it is causing a back pressure (full input, empty output)
> * By comparing exclusive/floating buffers usage, whether all channels are back-pressured or only some of them
> {code:java}
> public final class IOMetricsInfo {
> private final float outPoolUsage;
> private final float inputExclusiveBuffersUsage;
> private final float inputFloatingBuffersUsage;
> }
> {code}
> JobDetailsInfo.JobVertexDetailsInfo merge use Math.max.(ps: outPoolUsage is from upstream)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)