[jira] [Commented] (BAHIR-26) Add integration test infrastructure based on Docker

["Christian Kadner (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BAHIR-26?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553577#comment-15553577 ] 

Christian Kadner commented on BAHIR-26:
---------------------------------------

As far as I can figure, of the currently five Spark streaming connectors in Apache Bahir only {{streaming-zeromq}} which would require a dockerized test setup.

This is my current assessment:
 - {{sql-streaming-mqtt}}: the 3rd-party message broker ActiveMQ is included in test dependencies
 - {{streaming-akka}}: no 3rd-party tools required
 - {{streaming-mqtt}}: the 3rd-party message broker ActiveMQ is included in test dependencies
 - {{streaming-twitter}}: the 3rd-party here really is Twitter, I found no feasible way to isolate a mock Twitter server in a docker container to test with but we could easily test that a stream can be established and assert tweets are being consumed (assuming Twitter always has some traffic) or we could subscribe to our own twitter feed, generate our own tweets and assert we are consuming them
 - {{streaming-zeromq}}: requires native {{zmq}} libraries for test execution, the test code itself would have to run inside a docker container either by building and testing the Bahir ZeroMQ connector with maven inside the container or by mounting the local Bahir and Spark folders into a docker image and running some tests via Spark submit inside the container and then scrub the logs for success or failure




> Add integration test infrastructure based on Docker
> ---------------------------------------------------
>
>                 Key: BAHIR-26
>                 URL: https://issues.apache.org/jira/browse/BAHIR-26
>             Project: Bahir
>          Issue Type: New Feature
>            Reporter: Luciano Resende
>
> The existing Spark extensions are all a point of integration with a 3rd party tool such as a queue, etc. We want to start doing integration tests that actually simulate an environment that indeed communicates with these 3rd party tools to make sure we always have an end-to-end test working. In order to accomplish that, we could use docker images to launch the 3rd party tool involved on the integration tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)