You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Zhu Zhu (Jira)" <ji...@apache.org> on 2022/08/16 03:54:00 UTC
[jira] [Created] (FLINK-28981) Release Testing: Verify FLIP-245 sources speculative execution
Zhu Zhu created FLINK-28981:
-------------------------------
Summary: Release Testing: Verify FLIP-245 sources speculative execution
Key: FLINK-28981
URL: https://issues.apache.org/jira/browse/FLINK-28981
Project: Flink
Issue Type: Sub-task
Components: Connectors / Common, Runtime / Coordination
Reporter: Zhu Zhu
Fix For: 1.16.0
Speculative execution is introduced in Flink 1.16 to deal with temporary slow tasks caused by slow nodes. This feature currently consists of 4 FLIPs:
- FLIP-168: Speculative Execution core part
- FLIP-224: Blocklist Mechanism
- FLIP-245: Source Supports Speculative Execution
- FLIP-249: Flink Web UI Enhancement for Speculative Execution
This ticket aims for verifying FLIP-245, along with FLIP-168, FLIP-224 and FLIP-249.
More details about this feature and how to use it can be found in this documentation [PR|https://github.com/apache/flink/pull/20507].
To do the verification, the process can be:
- Write Flink jobs which has some {{source}} subtasks running much slower than others. 3 kinds of sources should be verified, including
- [Source functions|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java]
- [InputFormat sources|https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java]
- [FLIP-27 new sources|https://github.com/apache/flink/blob/master//flink-core/src/main/java/org/apache/flink/api/connector/source/Source.java]
- Modify Flink configuration file to enable speculative execution and tune the configuration as you like
- Submit the job. Checking the web UI, logs, metrics and produced result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)