[jira] [Comment Edited] (FLINK-30926) [Umbrella] Test Flink Release 1.17

["Hong Liang Teoh (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-30926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691301#comment-17691301 ] 

Hong Liang Teoh edited comment on FLINK-30926 at 2/20/23 9:26 PM:
------------------------------------------------------------------

h2. Test Outline

We want to submit a job that has parallelism > 1 and only 1 subtask is busy.

We want to verify FlameGraph correctly reflects the busy subtask.

 
h2. Test Setup

Flink job does the following:
 * Source from a Kinesis stream
 * Map with busy loop to calculate factorials
 * Discarding sink

 
{code:java}
public static long factorialUsingStreams(int n) {
    return LongStream.rangeClosed(1, n)
        .reduce(1, (long x, long y) -> x * y);
}

public static void main(String[] args) throws Exception {
  // ...
  env.addSource(new FlinkKinesisConsumer<>(
        STREAM, new SimpleStringSchema(), consumerConfig))
    .map((MapFunction<String, String>) value -> {
        for (int i = 0; i < 10; i++) {
            factorialUsingStreams(25);
        }
        return value;
    })
    .addSink(new DiscardingSink<>());
  env.execute("Busy loop");
}{code}
Flink job has parallelism of 4

Kinesis source stream only has 1 shard, so only 1 subtask will be reading records.

This means only 1 subtask will be busy.

 
h2. Results

Verified that the Flame graph accurately reflects this. See snapshots below

 


was (Author: JIRAUSER292614):
h2. Test Outline

We want to submit a job that has parallelism > 1 and only 1 subtask is busy.

We want to verify FlameGraph correctly reflects the busy subtask.

 
h2. Test Setup

Flink job does the following:
 * Source from a Kinesis stream
 * Map with busy loop to calculate factorials
 * Discarding sink

 
{code:java}
public static long factorialUsingStreams(int n) {
    return LongStream.rangeClosed(1, n)
        .reduce(1, (long x, long y) -> x * y);
}

public static void main(String[] args) throws Exception {
  // ...
  env.addSource(new FlinkKinesisConsumer<>(
        STREAM, new SimpleStringSchema(), consumerConfig))
    .map((MapFunction<String, String>) value -> {
        for (int i = 0; i < 10; i++) {
            factorialUsingStreams(25);
        }
        return value;
    })
    .addSink(new DiscardingSink<>());
  env.execute("Busy loop");
}{code}
Flink job has parallelism of 4

Kinesis source stream only has 1 shard, so only 1 subtask will be reading records.

This means only 1 subtask will be busy.

 

Verified that the Flame graph accurately reflects this. See snapshots below

 

> [Umbrella] Test Flink Release 1.17
> ----------------------------------
>
>                 Key: FLINK-30926
>                 URL: https://issues.apache.org/jira/browse/FLINK-30926
>             Project: Flink
>          Issue Type: Improvement
>          Components: Tests
>    Affects Versions: 1.17.0
>            Reporter: Qingsheng Ren
>            Priority: Blocker
>              Labels: release-testing
>             Fix For: 1.17.0
>
>
> This is an umbrella ticket for the Flink 1.17 testing efforts. Please prepare for the release testing by creating child testing tasks for the new features.
> Tickets for testing tasks should be opened with:
>  * Priority: Blocker
>  * Fix Version: 1.17.0
>  * Label: release-testing
> At the meantime, please update column value of `X-team verified` in the [1.17 Release Wiki page|https://cwiki.apache.org/confluence/display/FLINK/1.17+Release].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)