Stream API

[Siyuan Hua <si...@datatorrent.com>]

Hi community,

Apex, Spark Streaming and Flink, every stream processing framework has it's
own API to build a DAG. They all have pros and cons, but we also see many
similar concepts and APIs between Flink & Sparking Streaming. We want to
have this kind of similar APIs so that other developers can easily learn
and convert other apps to Apex.

Here is the link to the proposal
https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iHaVhmHQssY/edit#heading=h.aytn6rz7u1e4
Here is the ticket
https://issues.apache.org/jira/browse/APEXMALHAR-2006

Any suggestions are welcome.


Best,
Siyuan

Re: Stream API

[Thomas Weise <th...@datatorrent.com>]

+1 concise and intuitive way to define a DAG. Of course we will also need
to build out the underlying operators. What about iteration, join or window
based operations?


On Mon, Feb 29, 2016 at 9:05 PM, Bhupesh Chawda <bh...@datatorrent.com>
wrote:

> +1 for transforming Java 8 streams to an Apex Dag.
> This can act as an implementation of Java 8 streams for Big Data.
>
> -Bhupesh
>
>
> On Tue, Mar 1, 2016 at 8:18 AM, David Yan <da...@datatorrent.com> wrote:
>
> > I'm not a Java 8 expert, but it would be extremely valuable if we can
> > support the Java 8 stream API with by providing an implementation to
> > transform Java 8 streams to Apex topology.
> >
> > On Mon, Feb 29, 2016 at 3:30 PM, Siyuan Hua <si...@datatorrent.com>
> > wrote:
> >
> > > Hi community,
> > >
> > > Apex, Spark Streaming and Flink, every stream processing framework has
> > it's
> > > own API to build a DAG. They all have pros and cons, but we also see
> many
> > > similar concepts and APIs between Flink & Sparking Streaming. We want
> to
> > > have this kind of similar APIs so that other developers can easily
> learn
> > > and convert other apps to Apex.
> > >
> > > Here is the link to the proposal
> > >
> > >
> >
> https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iHaVhmHQssY/edit#heading=h.aytn6rz7u1e4
> > > Here is the ticket
> > > https://issues.apache.org/jira/browse/APEXMALHAR-2006
> > >
> > > Any suggestions are welcome.
> > >
> > >
> > > Best,
> > > Siyuan
> > >
> >
>

Re: Stream API

[Bhupesh Chawda <bh...@datatorrent.com>]

+1 for transforming Java 8 streams to an Apex Dag.
This can act as an implementation of Java 8 streams for Big Data.

-Bhupesh


On Tue, Mar 1, 2016 at 8:18 AM, David Yan <da...@datatorrent.com> wrote:

> I'm not a Java 8 expert, but it would be extremely valuable if we can
> support the Java 8 stream API with by providing an implementation to
> transform Java 8 streams to Apex topology.
>
> On Mon, Feb 29, 2016 at 3:30 PM, Siyuan Hua <si...@datatorrent.com>
> wrote:
>
> > Hi community,
> >
> > Apex, Spark Streaming and Flink, every stream processing framework has
> it's
> > own API to build a DAG. They all have pros and cons, but we also see many
> > similar concepts and APIs between Flink & Sparking Streaming. We want to
> > have this kind of similar APIs so that other developers can easily learn
> > and convert other apps to Apex.
> >
> > Here is the link to the proposal
> >
> >
> https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iHaVhmHQssY/edit#heading=h.aytn6rz7u1e4
> > Here is the ticket
> > https://issues.apache.org/jira/browse/APEXMALHAR-2006
> >
> > Any suggestions are welcome.
> >
> >
> > Best,
> > Siyuan
> >
>

Re: Stream API

[David Yan <da...@datatorrent.com>]

I'm not a Java 8 expert, but it would be extremely valuable if we can
support the Java 8 stream API with by providing an implementation to
transform Java 8 streams to Apex topology.

On Mon, Feb 29, 2016 at 3:30 PM, Siyuan Hua <si...@datatorrent.com> wrote:

> Hi community,
>
> Apex, Spark Streaming and Flink, every stream processing framework has it's
> own API to build a DAG. They all have pros and cons, but we also see many
> similar concepts and APIs between Flink & Sparking Streaming. We want to
> have this kind of similar APIs so that other developers can easily learn
> and convert other apps to Apex.
>
> Here is the link to the proposal
>
> https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iHaVhmHQssY/edit#heading=h.aytn6rz7u1e4
> Here is the ticket
> https://issues.apache.org/jira/browse/APEXMALHAR-2006
>
> Any suggestions are welcome.
>
>
> Best,
> Siyuan
>