[jira] [Commented] (FLINK-20114) Test Kafka Source based on the new Source API 

["Dong Lin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311175#comment-17311175 ] 

Dong Lin commented on FLINK-20114:
----------------------------------

Hi [~rmetzger], thank you for doing the test! Here is the status update.

The following patches have been developed to fix KafkaSource related bugs:

1) https://github.com/apache/flink/pull/15161 has been merged with multiple commits to fix the issues mentioned in this bug.

2) Jiangjie is working on https://github.com/becketqin/flink/commit/1bb2be0935d48153c67981476fc5046afa4b172f which can fix a bug that is known to prevent savepoint from completion.

After applying the patch mentioned below, I have run StateMachineExample and believe the issues 1-5 has been fixed. Please see below for details.

Note that for Issue 6, I could not reproduce the original issue and it is not clear which bug caused the original issue. So it may be useful to double check it with the test you used earlier.

1) Testing Issue 1:
This behavior is expected. If no Kafka broker was reachable, KafkaConsumer (and thus KafkaSource) is expected to keep polling without throwing error.

This behavior is reasonable because, when Kafka service goes offline due to either maintenance or failure, it is not helpful for KafkaConsumer to fail fast.

2) Testing Issue 2:
I could start StateMachineExample and then stop it with savepoint successfully.

3) Testing Issue 3:
The issue should be fixed by the commit https://github.com/apache/flink/commit/4c6c42392e7d116dee572fefb2e4e0e02abacefb

4) Testing Issue 4:
I could start StateMachineExample and then cancel it successfully. There is no ClassNotFoundException in log files.

5) Testing Issue 5:
The issue should be fixed by the commit https://github.com/apache/flink/commit/b23f65f17b4de16cf0fd91b225c3c3c61c849450

6) Testing Issue 6:
I could create the flink-demo-topic-1 topic with 64 partitions and then start StateMachineExample successfully. 



> Test Kafka Source based on the new Source API 
> ----------------------------------------------
>
>                 Key: FLINK-20114
>                 URL: https://issues.apache.org/jira/browse/FLINK-20114
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / Kafka
>    Affects Versions: 1.12.0
>            Reporter: Robert Metzger
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.13.0, 1.12.3
>
>         Attachments: Screenshot 2020-11-24 at 15.14.35.png, first-run.tgz, second-run.tgz
>
>
> Feature introduced in https://issues.apache.org/jira/browse/FLINK-18323
> -------- 
> [General Information about the Flink 1.12 release testing|https://cwiki.apache.org/confluence/display/FLINK/1.12+Release+-+Community+Testing]
> When testing a feature, consider the following aspects:
> - Is the documentation easy to understand
> - Are the error messages, log messages, APIs etc. easy to understand
> - Is the feature working as expected under normal conditions
> - Is the feature working / failing as expected with invalid input, induced errors etc.
> If you find a problem during testing, please file a ticket (Priority=Critical; Fix Version = 1.12.0), and link it in this testing ticket.
> During the testing, and once you are finished, please write a short summary of all things you have tested.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)