You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "文报 (Jira)" <ji...@apache.org> on 2022/03/31 05:36:00 UTC
[jira] [Updated] (FLINK-26938) HybridSource recovery from savepoint fails When flink parallelism is greater than the number of Kafka partitions
[ https://issues.apache.org/jira/browse/FLINK-26938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
文报 updated FLINK-26938:
-----------------------
Description:
HybridSource recovery from savepoint fails When flink parallelism is greater than the number of Kafka partitions
First test
Flink job before savePoint
flink parallelism =16
kafka partition=3
Flink after savePoint
case 1：
flink parallelism =16
kafka partition=3
HybridSource recovery from savepoint fails
!image-2022-03-31-11-12-56-562.png!
case 2：
flink parallelism =3
kafka partition=3
HybridSource recovery from savepoint successful
case 3：
flink parallelism =8
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
case 4：
flink parallelism =4
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
case 5：
flink parallelism =1
kafka partition=3
HybridSource recovery from savepoint successful
Second test
Flink job before savePoint
flink parallelism =3
kafka partition=3
Flink after savePoint
case 1：
flink parallelism =3
kafka partition=3
HybridSource recovery from savepoint successful
case 2：
flink parallelism =1
kafka partition=3
HybridSource recovery from savepoint successful
case 3：
flink parallelism =4
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
Specific code see the attached test code HybridSourceTest
was:
First test
Flink job before savePoint
flink parallelism =16
kafka partition=3
Flink after savePoint
case 1：
flink parallelism =16
kafka partition=3
HybridSource recovery from savepoint fails
!image-2022-03-31-11-12-56-562.png!
case 2：
flink parallelism =3
kafka partition=3
HybridSource recovery from savepoint successful
case 3：
flink parallelism =8
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
case 4：
flink parallelism =4
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
case 5：
flink parallelism =1
kafka partition=3
HybridSource recovery from savepoint successful
Second test
Flink job before savePoint
flink parallelism =3
kafka partition=3
Flink after savePoint
case 1：
flink parallelism =3
kafka partition=3
HybridSource recovery from savepoint successful
case 2：
flink parallelism =1
kafka partition=3
HybridSource recovery from savepoint successful
case 3：
flink parallelism =4
kafka partition=3
HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
Specific code see the attached test code HybridSourceTest
> HybridSource recovery from savepoint fails When flink parallelism is greater than the number of Kafka partitions
> ----------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-26938
> URL: https://issues.apache.org/jira/browse/FLINK-26938
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.14.0
> Environment: Flink 1.14.0
> Reporter: 文报
> Priority: Major
> Attachments: image-2022-03-31-13-25-28-520.png
>
>
> HybridSource recovery from savepoint fails When flink parallelism is greater than the number of Kafka partitions
> First test
> Flink job before savePoint
> flink parallelism =16
> kafka partition=3
> Flink after savePoint
> case 1：
> flink parallelism =16
> kafka partition=3
> HybridSource recovery from savepoint fails
> !image-2022-03-31-11-12-56-562.png!
> case 2：
> flink parallelism =3
> kafka partition=3
> HybridSource recovery from savepoint successful
> case 3：
> flink parallelism =8
> kafka partition=3
> HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
> case 4：
> flink parallelism =4
> kafka partition=3
> HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
> case 5：
> flink parallelism =1
> kafka partition=3
> HybridSource recovery from savepoint successful
> Second test
> Flink job before savePoint
> flink parallelism =3
> kafka partition=3
> Flink after savePoint
> case 1：
> flink parallelism =3
> kafka partition=3
> HybridSource recovery from savepoint successful
> case 2：
> flink parallelism =1
> kafka partition=3
> HybridSource recovery from savepoint successful
> case 3：
> flink parallelism =4
> kafka partition=3
> HybridSource recovery from savepoint fails the same NullPointerException: Source for index=0 not available
> Specific code see the attached test code HybridSourceTest
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)