You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/11/23 17:23:50 UTC

[GitHub] [flink-statefun] igalshilman opened a new pull request #178: [FLINK-20303][test] Add a SmokeE2E test

igalshilman opened a new pull request #178:
URL: https://github.com/apache/flink-statefun/pull/178


   #### This PR adds a "smoke" e2e test. This test goal is to mimic randomly behaving StateFun applications, and to surface bugs in the presence of failures.
   
   ##### Test description
   
   The test has a single parallelism source, that generates commands, in rapid succession, to be executed by a target function.
   Each target function, interprets the command and can do one of the following operations:
   - increment its state by 1
   - send a message to another function (message and address are also specified in the command)
   - send a message after a delay
   - register an asynchronous operation
   - send a message to an egress.
   
   (see `commands.proto`)
   Eventually the source switches to `verification` mode, where it sends, every 2 seconds a message to all the functions under test, a `Verification` command that contains the expected state that this function should have.
   Each function upon receiving this message, emits a `VerificationResult` into a specific egress.
   The test driver program will collect these verification results, and would complete the test if all the functions have their expected state. 
   
   During the generation phase, the source would throw an exception (`KABOOM`) to simulate a failure.
   
   The test driver bounds in time the execution of the test by 2 minutes, but this parameter would need to be tuned to avoid false positives in CI. 
   
   #### This test is designed to be run from:
   
   -  within test containers (to simulate distributed settings)
   -  via the `Harness` in the IDE for easier debuggability
   -  deployed in a real cluster.
   
   #### Parameters
   
   The test contains few parameters that control the randomness of the generated commands:
   ```
   ModuleParameters{
   numberOfFunctionInstances=128
    commandDepth=10
    messageCount=100000
    maxCommandsPerDepth=3
    stateModificationsPr=0.4
    sendPr=0.9
    sendAfterPr=0.1
    asyncSendPr=0.1
    noopPr=0.2
    sendEgressPr=0.03
    maxFailures=1
   }
   ```
   
   
   You can change the randomness of different operations to make it more or less likely of specific operations to occur.
   
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink-statefun] tzulitai closed pull request #178: [FLINK-20303][test] Add a SmokeE2E test

Posted by GitBox <gi...@apache.org>.
tzulitai closed pull request #178:
URL: https://github.com/apache/flink-statefun/pull/178


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink-statefun] tzulitai edited a comment on pull request #178: [FLINK-20303][test] Add a SmokeE2E test

Posted by GitBox <gi...@apache.org>.
tzulitai edited a comment on pull request #178:
URL: https://github.com/apache/flink-statefun/pull/178#issuecomment-737726878


   I had tried running this in a loop for 10 times in my local fork's Github Actions CI, and all attempts were successful.
   So it seems to be stable, and has proven to be useful by catching a few bugs already!
   
   Thanks for working on this @igalshilman, I'll proceed to merging this.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink-statefun] tzulitai commented on pull request #178: [FLINK-20303][test] Add a SmokeE2E test

Posted by GitBox <gi...@apache.org>.
tzulitai commented on pull request #178:
URL: https://github.com/apache/flink-statefun/pull/178#issuecomment-737726878


   I had tried running this in a loop for 10 times in my local fork's Github Actions CI, and all attempts were successful.
   This looks good to me, and has proven to be useful by catching a few bugs already!
   
   Thanks for working on this @igalshilman, I'll proceed to merging this.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org