[jira] [Commented] (FLINK-7343) Kafka010ProducerITCase instability

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406282#comment-16406282 ] 

ASF GitHub Bot commented on FLINK-7343:
---------------------------------------

GitHub user pnowojski opened a pull request:

    https://github.com/apache/flink/pull/5729

    [FLINK-7343][kafka-tests] Fix test at-least-once test instability

    This pr fixes instabilities in both Kafka 0.10 and Kafka 0.9.
    
    Previously we could set lastSnapshotedElement to some value during checkpointing AFTER executing shutdown while KafkaProducer snapshot of this value would fail. This was leading to incorrectly expect this value to be present in the test kafka topic. Fix is to remember lastSnapshotedElementBeforeShutdown - last snapshot that we exepct to succeed without failure.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pnowojski/flink f7343

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5729.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5729
    
----

----


> Kafka010ProducerITCase instability
> ----------------------------------
>
>                 Key: FLINK-7343
>                 URL: https://issues.apache.org/jira/browse/FLINK-7343
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>            Reporter: Piotr Nowojski
>            Assignee: Piotr Nowojski
>            Priority: Blocker
>              Labels: test-stability
>             Fix For: 1.5.0
>
>
> As reported by [~till.rohrmann] in https://issues.apache.org/jira/browse/FLINK-6996 there seems to be a test instability with `Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink`
> https://travis-ci.org/tillrohrmann/flink/jobs/258538641
> It is probably related to log.flush intervals in Kafka, which delay flushing the data to files and potentially causing data loses on killing Kafka brokers in the tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)