JUnit validation on cloud metrics during performance tests

[Andrew Burton <an...@gmail.com>]

Hi list,

Does anyone have experience in incorporating cloud metrics (typically AWS
or Azure) as part of their post-test JUnit validation whether via lightning
plugin - https://github.com/automatictester/lightning or some other Maven
plugin?

In addition to pass/fail tests on sampler response times or error rates, it
would be useful to add pass/fail conditions on such metrics as CPU
utilisation, queue/topic depth, etc. for the period of a test. In addition,
this means we could retain the relevant server stats in the results file
for retrospective analysis.

Most cloud providers provide a Java library to retrieve such stats - the
Azure one is here (
https://azure.microsoft.com/en-us/blog/manage-azure-monitor-using-java/).
The approach I had in mind was to:
* Add a tearDown thread group with a sampler to retrieve the relevant stats
for the duration of the test (e.g. the past hour)
* For each metric returned, convert this to a SampleResult object and log
to the results file
* Add a relevant JUnit test for the metrics

(An alternate option would be to have this as a Java sampler invoked at
5-10 minute intervals during the test).

Want to make sure I'm not re-inventing the wheel before I start work, and
whether anyone had other ideas or suggestions.

Thanks

Andrew