[jira] [Created] (BEAM-1027) Hosting data stores to enable IO Transform testing

["Stephen Sisk (JIRA)" <ji...@apache.org>]

Stephen Sisk created BEAM-1027:
----------------------------------

             Summary: Hosting data stores to enable IO Transform testing
                 Key: BEAM-1027
                 URL: https://issues.apache.org/jira/browse/BEAM-1027
             Project: Beam
          Issue Type: Task
            Reporter: Stephen Sisk


Currently we have a good set of unit tests for our IO Transforms - those
tend to run against in-memory versions of the data stores. However, we'd
like to further increase our test coverage to include running them against
real instances of the data stores that the IO Transforms work against (e.g.
cassandra, mongodb, kafka, etc…), which means we'll need to have real
instances of various data stores.

Additionally, if we want to do performance regression detection, it's
important to have instances of the services that behave realistically,
which isn't true of in-memory or dev versions of the services.

My proposed solution is in https://lists.apache.org/thread.html/367fd9669411f21c9ec1f2d27df60464f49d5ce81e6bd16de401d035@%3Cdev.beam.apache.org%3E 
- it still needs further discussion, and (assuming we agree on the general idea), the beam community needs to decide which cluster management software we want to use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)