Real time event processing - Storm Or Spark Streaming API OR Flink?

[Ravindra Babu <ra...@gmail.com>]

Hi,

I don't know whether it is right e-mail id to raise concerns or not. If not
please let me know right e-mail id.

Multiple projects are going on and with overlapping features. It leads
wastage of huge efforts.

Developers and end users are getting spoiled with more choices.  You should
look at consolidaiton of projects and get rid of duplicate efforts.

e.g. Real time event processing.

*Storm* is first choice for developers*.  *
*Spark Streaming *is recommended over  *Storm* after some time
*Flink *is recommended now.

When people are thinking of *Flink  *USP is real time event processing,
batch processing APIs are also available, which is again in parallel to
*Spark.*


Same is the case with *ActiveMQ *and *Kafka*

it's high time for consolidation of various projects and concentrate on key
projects.

Re: Real time event processing - Storm Or Spark Streaming API OR Flink?

[Shane Curcuru <as...@shanecurcuru.org>]

Ravindra Babu wrote on 3/5/16 6:17 PM:
> Hi,
> 
> I don't know whether it is right e-mail id to raise concerns or not. If not
> please let me know right e-mail id.

All work done on Apache projects is done by volunteers on those specific
projects.  The ASF as a whole does not provide technical direction, nor
do we push any sort of coordination between projects.

That is: work on Apache projects is done by the committers doing the
work themselves.  So if you have suggestions for specific Apache
projects to improve things, you need to show up with some specific
suggestions for improvements on their dev@ mailing list(s).

> Multiple projects are going on and with overlapping features. It leads
> wastage of huge efforts.
> 
> Developers and end users are getting spoiled with more choices.  You should
> look at consolidaiton of projects and get rid of duplicate efforts.
> 
> e.g. Real time event processing.
> 
> *Storm* is first choice for developers*.  *
> *Spark Streaming *is recommended over  *Storm* after some time
> *Flink *is recommended now.
> 
> When people are thinking of *Flink  *USP is real time event processing,
> batch processing APIs are also available, which is again in parallel to
> *Spark.*
> 
> 
> Same is the case with *ActiveMQ *and *Kafka*
> 
> it's high time for consolidation of various projects and concentrate on key
> projects.

More choices means that more different kinds of users might want to use
our software.  The mission of the ASF is to produce software for the
public good.  As long as *someone* thinks an Apache project is useful -
and is willing to contribute code back to that project - we're happy for
that project to go in whatever technical direction they want to.  Even
if that means there are several projects in similar technology spaces.

- Shane

Re: Real time event processing - Storm Or Spark Streaming API OR Flink?

[Gar Gyi <ga...@gmail.com>]

Help
On Mar 6, 2016 2:14 PM, "Ravindra Babu" <ra...@gmail.com> wrote:

> Hi,
>
> I don't know whether it is right e-mail id to raise concerns or not. If not
> please let me know right e-mail id.
>
> Multiple projects are going on and with overlapping features. It leads
> wastage of huge efforts.
>
> Developers and end users are getting spoiled with more choices.  You should
> look at consolidaiton of projects and get rid of duplicate efforts.
>
> e.g. Real time event processing.
>
> *Storm* is first choice for developers*.  *
> *Spark Streaming *is recommended over  *Storm* after some time
> *Flink *is recommended now.
>
> When people are thinking of *Flink  *USP is real time event processing,
> batch processing APIs are also available, which is again in parallel to
> *Spark.*
>
>
> Same is the case with *ActiveMQ *and *Kafka*
>
> it's high time for consolidation of various projects and concentrate on key
> projects.
>