[jira] [Created] (SPARK-5267) Add a streaming module to ingest Apache Camel Messages from a configured endpoints

["Steve Brewin (JIRA)" <ji...@apache.org>]

Steve Brewin created SPARK-5267:
-----------------------------------

             Summary: Add a streaming module to ingest Apache Camel Messages from a configured endpoints
                 Key: SPARK-5267
                 URL: https://issues.apache.org/jira/browse/SPARK-5267
             Project: Spark
          Issue Type: Improvement
          Components: Streaming
    Affects Versions: 1.2.0
            Reporter: Steve Brewin


The number of input stream protocols supported by Spark Streaming is quite limited, which constrains the number of systems with which it can be integrated.

This proposal solves the problem by adding an optional module that integrates Apache Camel, which support many more input protocols. Our tried and tested implementation of this proposal is "spark-streaming-camel". 

An Apache Camel service is run on a separate Thread, consuming each http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Message.html and storing it into Spark's memory. The provider of the Message is specified by any consuming component URI documented at http://camel.apache.org/components.html, making all of these protocols available to Spark Streaming.

Thoughts?






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org