[GitHub] jeremiaswerner opened a new pull request #3942: make error count and percentage configurable for gatling tests

[GitBox <gi...@apache.org>]

jeremiaswerner opened a new pull request #3942: make error count and percentage configurable for gatling tests
URL: https://github.com/apache/incubator-openwhisk/pull/3942
 
 
   <!--- Provide a concise summary of your changes in the Title -->
   
   ## Description
   Today the gatling tests hard code the amount and percentage of errors, like:
   https://github.com/apache/incubator-openwhisk/blob/master/tests/performance/gatling_tests/src/gatling/scala/ColdBlockingInvokeSimulation.scala#L88
   
   The PR is making the number of errors and percentage configurable per environment variable. For example during long running tests with lot of requests a certain amount of failure rate is acceptable.
   
   ## My changes affect the following components
   - [ ] API
   - [ ] Controller
   - [ ] Message Bus (e.g., Kafka)
   - [ ] Loadbalancer
   - [ ] Invoker
   - [ ] Intrinsic actions (e.g., sequences, conductors)
   - [ ] Data stores (e.g., CouchDB)
   - [x] Tests
   - [ ] Deployment
   - [ ] CLI
   - [ ] General tooling
   - [ ] Documentation
   
   ## Types of changes
   - [ ] Bug fix (generally a non-breaking change which closes an issue).
   - [x] Enhancement or new feature (adds new functionality).
   - [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
   
   ## Checklist:
   
   - [x] I signed an [Apache CLA](https://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md).
   - [x] I reviewed the [style guides](https://github.com/apache/incubator-openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :).
   - [ ] I added tests to cover my changes.
   - [ ] My changes require further changes to the documentation.
   - [x] I updated the documentation where necessary.
   
   I've run a test that the parameters are recognised as expected.
   ```
   OPENWHISK_HOST="xxxxxx" USERS="10" REQUESTS_PER_SEC="10" MAX_ERRORS_ALLOWED="1" MAX_ERRORS_ALLOWED_PERCENTAGE="0.1" ./gradlew gatlingRun-ColdBlockingInvokeSimulation
   ```
   
   Result
   ```
   Invoke action: mean requests per second is greater than 10.0 : true
   Invoke action: count of failed requests is less than or equal to 1.0 : true
   Invoke action: percentage of failed requests is less than or equal to 0.1 : true
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services