[jira] [Commented] (FLINK-8975) End-to-end test: Resume from savepoint

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407616#comment-16407616 ] 

ASF GitHub Bot commented on FLINK-8975:
---------------------------------------

GitHub user tzulitai opened a pull request:

    https://github.com/apache/flink/pull/5733

    [FLINK-8975] [test] Add resume from savepoint end-to-end test

    ## What is the purpose of the change
    
    This pull request adds an end-to-end test that verifies resuming a job from a savepoint.
    
    The complete end-to-end test consists of the following:
    
    1. The `StateMachineExample` is used for the end-to-end test
    2. A separate job to generate Kafka events for the state machine is run
    3. After the state machine job runs for a while, we take a savepoint.
    4. Cancelling and resuming the state machine example job with the savepoint.
    
    All the above steps should not result in any errors or outputs from the state machine job. If so, the end-to-end test would fail.
    
    ## Brief change log
    
    - Add a separate main class for the Kafka events generator job
    - Add `test_resume_savepoint.sh` test script
    
    ## Verifying this change
    
    This PR itself introduces a new test.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tzulitai/flink FLINK-8975

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5733.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5733
    
----
commit 529e060cb05fd723b8656dcc9ef48f8011282dd8
Author: Tzu-Li (Gordon) Tai <tz...@...>
Date:   2018-03-21T08:25:37Z

    [FLINK-8975] [test] Add Kafka events generator job for StateMachineExample

commit 213638b4194cceccd597e90c78631a6c6a191abb
Author: Tzu-Li (Gordon) Tai <tz...@...>
Date:   2018-03-21T08:32:51Z

    [FLINK-8975] [test] Add resume from savepoint end-to-end test

----


> End-to-end test: Resume from savepoint
> --------------------------------------
>
>                 Key: FLINK-8975
>                 URL: https://issues.apache.org/jira/browse/FLINK-8975
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 1.5.0
>            Reporter: Till Rohrmann
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Blocker
>             Fix For: 1.5.0
>
>
> User usually take a savepoint and want to resume from it. In order to verify that Flink supports this feature, we should add an end-to-end test which scripts this behavior. We should use the general purpose testing job FLINK-8971 with failures disabled for that.
> The end-to-end test should do the following:
> * Submit FLINK-8971 job
> * Verify that the savepoint is there
> * Cancel job and resume from savepoint
> * Verify that job could be resumed
> * Use different StateBackends: RocksDB incremental async/sync, RocksDB full async/sync, FsStateBackend aysnc/sync



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)