[jira] [Created] (STORM-1508) Investigate creating integration test framework

["Paul Poulosky (JIRA)" <ji...@apache.org>]

Paul Poulosky created STORM-1508:
------------------------------------

             Summary: Investigate creating integration test framework
                 Key: STORM-1508
                 URL: https://issues.apache.org/jira/browse/STORM-1508
             Project: Apache Storm
          Issue Type: Story
            Reporter: Paul Poulosky


This is related to STORM-1179, and an offline discussion.

It would be helpful if we could have an integration test framework that could deploy storm via standard tools and run true integration tests.  This would be useful for test scenarios that cannot be tested with LocalCluster.

The test framework would need to

1)  node / VM allocation
2)  deployment
3)  modify configuration
4)  run tests / report results

Ducktape (https://github.com/confluentinc/ducktape/wiki) has been suggested as a common framework we could use to perform this type of testing.


>From offline discussion:

Yes we too have similar tests based on internal Yahoo testing.  Really it comes down to a few operations that the test framework needs to be able to do 1) node/VM allocation 2) deployment 3) modify configuration/processes on running nodes 4) run actual tests.
The first three are often specific to the setup.  Is it open stack?  Amazon EC2?  Are we using ssh to go between nodes, or is it something else?  All of this is something that we should be able to accomplish with an abstraction layer.  ducktape appears to have a fairly decent abstraction layer for the first even allowing for a static JSON config for the hosts.  The only other requirement seems to be ssh access between the nodes so it can run various python scripts.  As we are developing these tests we probably want to be sure that we have a pluggable abstraction for the other layers as well.  That way if I am using RPMs and Chef to do a deployment and be sure my RPM packaging is correct I can do that, and if someone else just wants to take the TGZ and scp it everywhere that works too.
I also hope that we can limit the score of these tests.  The current Yahoo integration tests take several hours to run because we are constantly trying to reconfigure the cluster to test different setups.  Ideally we can test failure scenarios and basic integration with other tools, but anything that can be done on a LocalCluster should probably be tested there, to reduce the total running time and debugging complexity.
- Bobby



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)