You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hong Liang Teoh (Jira)" <ji...@apache.org> on 2023/02/20 21:35:00 UTC
[jira] [Commented] (FLINK-31159) Release Testing: Verify FLINK-30583 Provide the flame graph to the subtask level
[ https://issues.apache.org/jira/browse/FLINK-31159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691304#comment-17691304 ]
Hong Liang Teoh commented on FLINK-31159:
-----------------------------------------
h2. Test Outline
We want to submit a job that has parallelism > 1 and only 1 subtask is busy.
We want to verify FlameGraph correctly reflects the busy subtask.
h2. Test Setup
Flink job does the following:
* Source from a Kinesis stream
* Map with busy loop to calculate factorials
* Discarding sink
{code:java}
public static long factorialUsingStreams(int n) {
return LongStream.rangeClosed(1, n)
.reduce(1, (long x, long y) -> x * y);
}
public static void main(String[] args) throws Exception {
// ...
env.addSource(new FlinkKinesisConsumer<>(
STREAM, new SimpleStringSchema(), consumerConfig))
.map((MapFunction<String, String>) value -> {
for (int i = 0; i < 10; i++) {
factorialUsingStreams(25);
}
return value;
})
.addSink(new DiscardingSink<>());
env.execute("Busy loop");
}{code}
Flink job has parallelism of 4
Kinesis source stream only has 1 shard, so only 1 subtask will be reading records.
This means only 1 subtask will be busy.
h2. Results
Verified that the Flame graph accurately reflects this. See snapshots below
- Only subtask 3 has records
!Map_record_distribution.png!
* Flame graph with "all" setting
!all_subtasks.png!
* FlameGraph with subtask 0 shows nothing on CPU
!subtask0.png!
* FlameGraph from subtask 3 shows CPU usage as shown on "all"
!subtask3.png!
> Release Testing: Verify FLINK-30583 Provide the flame graph to the subtask level
> --------------------------------------------------------------------------------
>
> Key: FLINK-31159
> URL: https://issues.apache.org/jira/browse/FLINK-31159
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / REST, Runtime / Web Frontend
> Affects Versions: 1.17.0
> Reporter: Hong Liang Teoh
> Priority: Major
> Fix For: 1.17.0
>
> Attachments: Map_record_distribution.png, all_subtasks.png, subtask0.png, subtask3.png
>
>
> The issue aims to verify FLINK-30583.
> Please verify:
> # When below conditions are met:
> ** Job has more than 1 parallel subtask
> ** Some subtasks are busy, whilst others are not (e.g. some receive records, some don't)
> ** The FlameGraph accurately reflects the busy subtask
--
This message was sent by Atlassian Jira
(v8.20.10#820010)