You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Raul Kripalani <ra...@apache.org> on 2015/09/20 14:37:11 UTC
Brainstorming about Reactive Streams
Hey Igniters,
Lalit Kumar Jha expressed interest [1] in contributing to ticket IGNITE-815
[2] which aims to bring in Reactive Streams to Ignite.
I wanted to start a brainstorm about which functionalities we'd like to
cover.
Off the top of my head, I can think of merging, filtering, sorting,
batching, grouping, etc. on Data Streams, Ignite Messages and Events. Also,
integration with Continuous Queries would be useful.
Furthermore, I wanted to ask the community if at this point we feel
comfortable with Reactive Streams per-se or we prefer to go with RxJava for
now. Reactive Streams is a proposed standard whose main Java implementation
and is slowly gaining traction (e.g. vert.x [3] and MongoDB integrations
[4]), whereas RxJava is an already mature and production-tested
implementation of the initial ReactiveX design by Microsoft [5] with 7000+
stars on GitHub.
RxJava is the foundation of Hystrix [6], a powerful library for latency and
fault tolerance management. RxJava does not implement Reactive Streams as
of 1.x (as they predated it), but they've vowed support for 2.x [7].
Moreover, there's already an adapter which passes the TCK [8].
My take is as follows. I'd like to follow the Reactive Streams standard,
but I wouldn't like to lose support for Hystrix (hopefully it'll eventually
be migrated to Reactive Streams too). So an option is to go with the RxJava
+ adapter design for now to get the best of both worlds. Once RxJava 2.0 is
out, we can drop the adapter out of the picture.
Thoughts?
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
[2] https://issues.apache.org/jira/browse/IGNITE-815
[3] http://vertx.io/docs/vertx-reactive-streams/java/
[4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
[5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
[6] https://github.com/Netflix/Hystrix
[7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
[8] https://github.com/ReactiveX/RxJavaReactiveStreams
Regards,
*Raúl Kripalani*
Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk
Re: Brainstorming about Reactive Streams
Posted by Lalit Kumar Jha <la...@gmail.com>.
Yes, we should not modify existing API, to support backward compatability
and richer functionality (the primary goal of product).
It will be additional wrapper/interface available.
This will also does not break anything and we can migrate slowly.
On Wed, Sep 23, 2015 at 9:08 PM, Sergi Vladykin <se...@gmail.com>
wrote:
> Lalit,
>
> I don't think we are going to change our APIs to some 3rd party standards
> or even worse to some 3rd party library.
> There is already JCache from JSR-107 and I can't say we are 100% happy with
> it.
>
> Thus I can consider Reactive Streams to be an additional optional API but
> not
> `All data flow/stream in system adhere to “Reactive Streams” API` as you
> said.
>
> Sergi
>
>
>
> 2015-09-23 18:02 GMT+03:00 Gianfranco Murador <
> murador.gianfranco@gmail.com>
> :
>
> > It ' a nice feature to be included in the grid compute module , in my
> > opinion . But I have a question : what difference we have with
> ComputeTask
> > class in behavioral terms ?
> >
> > https://apacheignite.readme.io/docs/compute-tasks
> >
> > 2015-09-23 16:45 GMT+02:00 Lalit Kumar Jha <la...@gmail.com>:
> >
> > > Summarizing this:
> > >
> > > *Goal*: All data flow/stream in system adhere to “Reactive Streams”
> API,
> > > includes data ingestion, async computation, compute result/stream
> > > aggregation.
> > >
> > > *Benefits*:
> > >
> > >
> > > - Common interface for all data transfer/exchange/processing.
> > > - Streaming data ingestion can compose complex stream processing in
> a
> > > standard (interoperable dropin implementation replacement) fashion,
> > like
> > > filter or aggregate streaming data, merge multiple stream etc.
> > >
> > >
> > > - Compose multiple async compute jobs results as streams.
> > >
> > >
> > > *Design*: To overcome “Cons of Exposing Reactive Stream APIs instead of
> > > RxJava (or other implementation)” mentioned at
> > > https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> > >
> > > #1. Follow recommended approach there.
> > >
> > >
> > > #2. Rather than having streaming (data transfer) stream processing
> logic
> > > (filter, merge, flatMap) in Consumer, treat them as a separate concern,
> > and
> > > wrap Publisher/Consumer in concrete classes.
> > >
> > > This is discouraged by RxJava, but will keep standard and non standard
> > > implementation separate and will allow using multiple libraries for
> > > different use-cases. Rather than having this behaviour tightly coupled
> in
> > > Consumer.
> > >
> > >
> > >
> > > On Wed, Sep 23, 2015 at 6:29 PM, Raul Kripalani <ra...@apache.org>
> > wrote:
> > >
> > > > Yeah, in my original post I already pointed out that Reactive Streams
> > is
> > > a
> > > > proposed standard but not an implementation.
> > > >
> > > > My proposal was to go with RxJava 1.0 + the Reactive Streams adapter
> (
> > > > https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that
> > > would
> > > > allow users to benefit from things like Hystrix whose base is RxJava
> > 1.0,
> > > > while keeping their code future-proof for when RxJava 2.0 is released
> > > with
> > > > a native RS interface.
> > > >
> > > > Thanks for chiming in.
> > > >
> > > > Raúl.
> > > >
> > > > On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com>
> wrote:
> > > >
> > > > > I'd definitely go on a Reactive-Streams path (although I don't know
> > > what
> > > > > Apache Ignite does), however, there seems to be a small confusion
> > > around
> > > > > RS.
> > > > > RS is a set of 4 interfaces and practically unusable on its own.
> One
> > > > needs
> > > > > a
> > > > > library built around it such as Project Reactor, Akka Streams and
> > > RxJava
> > > > > 2.0. Such library, likely, will also slip into the API itself due
> to
> > > its
> > > > > usefulness.
> > > > >
> > > > > For example, the following API is inconvenient to use but is
> > definitely
> > > > > cutting-edge:
> > > > >
> > > > > Publisher<Data> getData();
> > > > >
> > > > > but you can only subscribe to it in its plain form or wrap it with
> > one
> > > of
> > > > > the libraries:
> > > > >
> > > > > Observable.fromPublisher(getData()).map().filter().group()...
> > > > >
> > > > > I'm not sure about the others, but RxJava 2.0 Observable (which
> later
> > > may
> > > > > be
> > > > > renamed to Flowable) implement Publisher directly, therefore, the
> > > > following
> > > > > getData() may return an RxJava Observable and still be consumable
> via
> > > > > Reactor or Akka (after wrapping/fluent-conversion, of course):
> > > > >
> > > > > getData().to(Streams::from).dispatchOn(...)
> > > > >
> > > > > Although I'm pretty confident that RxJava 2.0 works right now, it
> is
> > > > still
> > > > > considered alpha and based on past experience with its gatekeepers,
> > it
> > > > may
> > > > > take 6-12 months until an official release comes out.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > View this message in context:
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> > > > > Sent from the Apache Ignite Developers mailing list archive at
> > > > Nabble.com.
> > > > >
> > > >
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Sergi Vladykin <se...@gmail.com>.
Lalit,
I don't think we are going to change our APIs to some 3rd party standards
or even worse to some 3rd party library.
There is already JCache from JSR-107 and I can't say we are 100% happy with
it.
Thus I can consider Reactive Streams to be an additional optional API but
not
`All data flow/stream in system adhere to “Reactive Streams” API` as you
said.
Sergi
2015-09-23 18:02 GMT+03:00 Gianfranco Murador <mu...@gmail.com>
:
> It ' a nice feature to be included in the grid compute module , in my
> opinion . But I have a question : what difference we have with ComputeTask
> class in behavioral terms ?
>
> https://apacheignite.readme.io/docs/compute-tasks
>
> 2015-09-23 16:45 GMT+02:00 Lalit Kumar Jha <la...@gmail.com>:
>
> > Summarizing this:
> >
> > *Goal*: All data flow/stream in system adhere to “Reactive Streams” API,
> > includes data ingestion, async computation, compute result/stream
> > aggregation.
> >
> > *Benefits*:
> >
> >
> > - Common interface for all data transfer/exchange/processing.
> > - Streaming data ingestion can compose complex stream processing in a
> > standard (interoperable dropin implementation replacement) fashion,
> like
> > filter or aggregate streaming data, merge multiple stream etc.
> >
> >
> > - Compose multiple async compute jobs results as streams.
> >
> >
> > *Design*: To overcome “Cons of Exposing Reactive Stream APIs instead of
> > RxJava (or other implementation)” mentioned at
> > https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> >
> > #1. Follow recommended approach there.
> >
> >
> > #2. Rather than having streaming (data transfer) stream processing logic
> > (filter, merge, flatMap) in Consumer, treat them as a separate concern,
> and
> > wrap Publisher/Consumer in concrete classes.
> >
> > This is discouraged by RxJava, but will keep standard and non standard
> > implementation separate and will allow using multiple libraries for
> > different use-cases. Rather than having this behaviour tightly coupled in
> > Consumer.
> >
> >
> >
> > On Wed, Sep 23, 2015 at 6:29 PM, Raul Kripalani <ra...@apache.org>
> wrote:
> >
> > > Yeah, in my original post I already pointed out that Reactive Streams
> is
> > a
> > > proposed standard but not an implementation.
> > >
> > > My proposal was to go with RxJava 1.0 + the Reactive Streams adapter (
> > > https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that
> > would
> > > allow users to benefit from things like Hystrix whose base is RxJava
> 1.0,
> > > while keeping their code future-proof for when RxJava 2.0 is released
> > with
> > > a native RS interface.
> > >
> > > Thanks for chiming in.
> > >
> > > Raúl.
> > >
> > > On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com> wrote:
> > >
> > > > I'd definitely go on a Reactive-Streams path (although I don't know
> > what
> > > > Apache Ignite does), however, there seems to be a small confusion
> > around
> > > > RS.
> > > > RS is a set of 4 interfaces and practically unusable on its own. One
> > > needs
> > > > a
> > > > library built around it such as Project Reactor, Akka Streams and
> > RxJava
> > > > 2.0. Such library, likely, will also slip into the API itself due to
> > its
> > > > usefulness.
> > > >
> > > > For example, the following API is inconvenient to use but is
> definitely
> > > > cutting-edge:
> > > >
> > > > Publisher<Data> getData();
> > > >
> > > > but you can only subscribe to it in its plain form or wrap it with
> one
> > of
> > > > the libraries:
> > > >
> > > > Observable.fromPublisher(getData()).map().filter().group()...
> > > >
> > > > I'm not sure about the others, but RxJava 2.0 Observable (which later
> > may
> > > > be
> > > > renamed to Flowable) implement Publisher directly, therefore, the
> > > following
> > > > getData() may return an RxJava Observable and still be consumable via
> > > > Reactor or Akka (after wrapping/fluent-conversion, of course):
> > > >
> > > > getData().to(Streams::from).dispatchOn(...)
> > > >
> > > > Although I'm pretty confident that RxJava 2.0 works right now, it is
> > > still
> > > > considered alpha and based on past experience with its gatekeepers,
> it
> > > may
> > > > take 6-12 months until an official release comes out.
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> > > > Sent from the Apache Ignite Developers mailing list archive at
> > > Nabble.com.
> > > >
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Lalit Kumar Jha <la...@gmail.com>.
Gianfranco,
Behaviour wise it is similar. In this, data (streams) processed by map into
new stream and this new stream is processed by reduce stream.
In RS, multiple such opeartions are composed. This is as per MapReduce, RS
can exist alongside, by two means:
1. A new entity (reactive) in compute section.
2. ComputeTask implementation, adapting RS as per MapReduce semantics.
On Wed, Sep 23, 2015 at 8:32 PM, Gianfranco Murador <
murador.gianfranco@gmail.com> wrote:
> It ' a nice feature to be included in the grid compute module , in my
> opinion . But I have a question : what difference we have with ComputeTask
> class in behavioral terms ?
>
> https://apacheignite.readme.io/docs/compute-tasks
>
> 2015-09-23 16:45 GMT+02:00 Lalit Kumar Jha <la...@gmail.com>:
>
> > Summarizing this:
> >
> > *Goal*: All data flow/stream in system adhere to “Reactive Streams” API,
> > includes data ingestion, async computation, compute result/stream
> > aggregation.
> >
> > *Benefits*:
> >
> >
> > - Common interface for all data transfer/exchange/processing.
> > - Streaming data ingestion can compose complex stream processing in a
> > standard (interoperable dropin implementation replacement) fashion,
> like
> > filter or aggregate streaming data, merge multiple stream etc.
> >
> >
> > - Compose multiple async compute jobs results as streams.
> >
> >
> > *Design*: To overcome “Cons of Exposing Reactive Stream APIs instead of
> > RxJava (or other implementation)” mentioned at
> > https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> >
> > #1. Follow recommended approach there.
> >
> >
> > #2. Rather than having streaming (data transfer) stream processing logic
> > (filter, merge, flatMap) in Consumer, treat them as a separate concern,
> and
> > wrap Publisher/Consumer in concrete classes.
> >
> > This is discouraged by RxJava, but will keep standard and non standard
> > implementation separate and will allow using multiple libraries for
> > different use-cases. Rather than having this behaviour tightly coupled in
> > Consumer.
> >
> >
> >
> > On Wed, Sep 23, 2015 at 6:29 PM, Raul Kripalani <ra...@apache.org>
> wrote:
> >
> > > Yeah, in my original post I already pointed out that Reactive Streams
> is
> > a
> > > proposed standard but not an implementation.
> > >
> > > My proposal was to go with RxJava 1.0 + the Reactive Streams adapter (
> > > https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that
> > would
> > > allow users to benefit from things like Hystrix whose base is RxJava
> 1.0,
> > > while keeping their code future-proof for when RxJava 2.0 is released
> > with
> > > a native RS interface.
> > >
> > > Thanks for chiming in.
> > >
> > > Raúl.
> > >
> > > On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com> wrote:
> > >
> > > > I'd definitely go on a Reactive-Streams path (although I don't know
> > what
> > > > Apache Ignite does), however, there seems to be a small confusion
> > around
> > > > RS.
> > > > RS is a set of 4 interfaces and practically unusable on its own. One
> > > needs
> > > > a
> > > > library built around it such as Project Reactor, Akka Streams and
> > RxJava
> > > > 2.0. Such library, likely, will also slip into the API itself due to
> > its
> > > > usefulness.
> > > >
> > > > For example, the following API is inconvenient to use but is
> definitely
> > > > cutting-edge:
> > > >
> > > > Publisher<Data> getData();
> > > >
> > > > but you can only subscribe to it in its plain form or wrap it with
> one
> > of
> > > > the libraries:
> > > >
> > > > Observable.fromPublisher(getData()).map().filter().group()...
> > > >
> > > > I'm not sure about the others, but RxJava 2.0 Observable (which later
> > may
> > > > be
> > > > renamed to Flowable) implement Publisher directly, therefore, the
> > > following
> > > > getData() may return an RxJava Observable and still be consumable via
> > > > Reactor or Akka (after wrapping/fluent-conversion, of course):
> > > >
> > > > getData().to(Streams::from).dispatchOn(...)
> > > >
> > > > Although I'm pretty confident that RxJava 2.0 works right now, it is
> > > still
> > > > considered alpha and based on past experience with its gatekeepers,
> it
> > > may
> > > > take 6-12 months until an official release comes out.
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> > > > Sent from the Apache Ignite Developers mailing list archive at
> > > Nabble.com.
> > > >
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Gianfranco Murador <mu...@gmail.com>.
It ' a nice feature to be included in the grid compute module , in my
opinion . But I have a question : what difference we have with ComputeTask
class in behavioral terms ?
https://apacheignite.readme.io/docs/compute-tasks
2015-09-23 16:45 GMT+02:00 Lalit Kumar Jha <la...@gmail.com>:
> Summarizing this:
>
> *Goal*: All data flow/stream in system adhere to “Reactive Streams” API,
> includes data ingestion, async computation, compute result/stream
> aggregation.
>
> *Benefits*:
>
>
> - Common interface for all data transfer/exchange/processing.
> - Streaming data ingestion can compose complex stream processing in a
> standard (interoperable dropin implementation replacement) fashion, like
> filter or aggregate streaming data, merge multiple stream etc.
>
>
> - Compose multiple async compute jobs results as streams.
>
>
> *Design*: To overcome “Cons of Exposing Reactive Stream APIs instead of
> RxJava (or other implementation)” mentioned at
> https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
>
> #1. Follow recommended approach there.
>
>
> #2. Rather than having streaming (data transfer) stream processing logic
> (filter, merge, flatMap) in Consumer, treat them as a separate concern, and
> wrap Publisher/Consumer in concrete classes.
>
> This is discouraged by RxJava, but will keep standard and non standard
> implementation separate and will allow using multiple libraries for
> different use-cases. Rather than having this behaviour tightly coupled in
> Consumer.
>
>
>
> On Wed, Sep 23, 2015 at 6:29 PM, Raul Kripalani <ra...@apache.org> wrote:
>
> > Yeah, in my original post I already pointed out that Reactive Streams is
> a
> > proposed standard but not an implementation.
> >
> > My proposal was to go with RxJava 1.0 + the Reactive Streams adapter (
> > https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that
> would
> > allow users to benefit from things like Hystrix whose base is RxJava 1.0,
> > while keeping their code future-proof for when RxJava 2.0 is released
> with
> > a native RS interface.
> >
> > Thanks for chiming in.
> >
> > Raúl.
> >
> > On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com> wrote:
> >
> > > I'd definitely go on a Reactive-Streams path (although I don't know
> what
> > > Apache Ignite does), however, there seems to be a small confusion
> around
> > > RS.
> > > RS is a set of 4 interfaces and practically unusable on its own. One
> > needs
> > > a
> > > library built around it such as Project Reactor, Akka Streams and
> RxJava
> > > 2.0. Such library, likely, will also slip into the API itself due to
> its
> > > usefulness.
> > >
> > > For example, the following API is inconvenient to use but is definitely
> > > cutting-edge:
> > >
> > > Publisher<Data> getData();
> > >
> > > but you can only subscribe to it in its plain form or wrap it with one
> of
> > > the libraries:
> > >
> > > Observable.fromPublisher(getData()).map().filter().group()...
> > >
> > > I'm not sure about the others, but RxJava 2.0 Observable (which later
> may
> > > be
> > > renamed to Flowable) implement Publisher directly, therefore, the
> > following
> > > getData() may return an RxJava Observable and still be consumable via
> > > Reactor or Akka (after wrapping/fluent-conversion, of course):
> > >
> > > getData().to(Streams::from).dispatchOn(...)
> > >
> > > Although I'm pretty confident that RxJava 2.0 works right now, it is
> > still
> > > considered alpha and based on past experience with its gatekeepers, it
> > may
> > > take 6-12 months until an official release comes out.
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> > > Sent from the Apache Ignite Developers mailing list archive at
> > Nabble.com.
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Lalit Kumar Jha <la...@gmail.com>.
Summarizing this:
*Goal*: All data flow/stream in system adhere to “Reactive Streams” API,
includes data ingestion, async computation, compute result/stream
aggregation.
*Benefits*:
- Common interface for all data transfer/exchange/processing.
- Streaming data ingestion can compose complex stream processing in a
standard (interoperable dropin implementation replacement) fashion, like
filter or aggregate streaming data, merge multiple stream etc.
- Compose multiple async compute jobs results as streams.
*Design*: To overcome “Cons of Exposing Reactive Stream APIs instead of
RxJava (or other implementation)” mentioned at
https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
#1. Follow recommended approach there.
#2. Rather than having streaming (data transfer) stream processing logic
(filter, merge, flatMap) in Consumer, treat them as a separate concern, and
wrap Publisher/Consumer in concrete classes.
This is discouraged by RxJava, but will keep standard and non standard
implementation separate and will allow using multiple libraries for
different use-cases. Rather than having this behaviour tightly coupled in
Consumer.
On Wed, Sep 23, 2015 at 6:29 PM, Raul Kripalani <ra...@apache.org> wrote:
> Yeah, in my original post I already pointed out that Reactive Streams is a
> proposed standard but not an implementation.
>
> My proposal was to go with RxJava 1.0 + the Reactive Streams adapter (
> https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that would
> allow users to benefit from things like Hystrix whose base is RxJava 1.0,
> while keeping their code future-proof for when RxJava 2.0 is released with
> a native RS interface.
>
> Thanks for chiming in.
>
> Raúl.
>
> On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com> wrote:
>
> > I'd definitely go on a Reactive-Streams path (although I don't know what
> > Apache Ignite does), however, there seems to be a small confusion around
> > RS.
> > RS is a set of 4 interfaces and practically unusable on its own. One
> needs
> > a
> > library built around it such as Project Reactor, Akka Streams and RxJava
> > 2.0. Such library, likely, will also slip into the API itself due to its
> > usefulness.
> >
> > For example, the following API is inconvenient to use but is definitely
> > cutting-edge:
> >
> > Publisher<Data> getData();
> >
> > but you can only subscribe to it in its plain form or wrap it with one of
> > the libraries:
> >
> > Observable.fromPublisher(getData()).map().filter().group()...
> >
> > I'm not sure about the others, but RxJava 2.0 Observable (which later may
> > be
> > renamed to Flowable) implement Publisher directly, therefore, the
> following
> > getData() may return an RxJava Observable and still be consumable via
> > Reactor or Akka (after wrapping/fluent-conversion, of course):
> >
> > getData().to(Streams::from).dispatchOn(...)
> >
> > Although I'm pretty confident that RxJava 2.0 works right now, it is
> still
> > considered alpha and based on past experience with its gatekeepers, it
> may
> > take 6-12 months until an official release comes out.
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> > Sent from the Apache Ignite Developers mailing list archive at
> Nabble.com.
> >
>
Re: Brainstorming about Reactive Streams
Posted by Raul Kripalani <ra...@apache.org>.
Yeah, in my original post I already pointed out that Reactive Streams is a
proposed standard but not an implementation.
My proposal was to go with RxJava 1.0 + the Reactive Streams adapter (
https://github.com/ReactiveX/RxJavaReactiveStreams) for now, as that would
allow users to benefit from things like Hystrix whose base is RxJava 1.0,
while keeping their code future-proof for when RxJava 2.0 is released with
a native RS interface.
Thanks for chiming in.
Raúl.
On Wed, Sep 23, 2015 at 1:35 PM, akarnokd <ak...@gmail.com> wrote:
> I'd definitely go on a Reactive-Streams path (although I don't know what
> Apache Ignite does), however, there seems to be a small confusion around
> RS.
> RS is a set of 4 interfaces and practically unusable on its own. One needs
> a
> library built around it such as Project Reactor, Akka Streams and RxJava
> 2.0. Such library, likely, will also slip into the API itself due to its
> usefulness.
>
> For example, the following API is inconvenient to use but is definitely
> cutting-edge:
>
> Publisher<Data> getData();
>
> but you can only subscribe to it in its plain form or wrap it with one of
> the libraries:
>
> Observable.fromPublisher(getData()).map().filter().group()...
>
> I'm not sure about the others, but RxJava 2.0 Observable (which later may
> be
> renamed to Flowable) implement Publisher directly, therefore, the following
> getData() may return an RxJava Observable and still be consumable via
> Reactor or Akka (after wrapping/fluent-conversion, of course):
>
> getData().to(Streams::from).dispatchOn(...)
>
> Although I'm pretty confident that RxJava 2.0 works right now, it is still
> considered alpha and based on past experience with its gatekeepers, it may
> take 6-12 months until an official release comes out.
>
>
>
> --
> View this message in context:
> http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>
Re: Brainstorming about Reactive Streams
Posted by akarnokd <ak...@gmail.com>.
I'd definitely go on a Reactive-Streams path (although I don't know what
Apache Ignite does), however, there seems to be a small confusion around RS.
RS is a set of 4 interfaces and practically unusable on its own. One needs a
library built around it such as Project Reactor, Akka Streams and RxJava
2.0. Such library, likely, will also slip into the API itself due to its
usefulness.
For example, the following API is inconvenient to use but is definitely
cutting-edge:
Publisher<Data> getData();
but you can only subscribe to it in its plain form or wrap it with one of
the libraries:
Observable.fromPublisher(getData()).map().filter().group()...
I'm not sure about the others, but RxJava 2.0 Observable (which later may be
renamed to Flowable) implement Publisher directly, therefore, the following
getData() may return an RxJava Observable and still be consumable via
Reactor or Akka (after wrapping/fluent-conversion, of course):
getData().to(Streams::from).dispatchOn(...)
Although I'm pretty confident that RxJava 2.0 works right now, it is still
considered alpha and based on past experience with its gatekeepers, it may
take 6-12 months until an official release comes out.
--
View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Brainstorming-about-Reactive-Streams-tp3346p3422.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Brainstorming about Reactive Streams
Posted by Raul Kripalani <ra...@apache.org>.
Hi,
First things first, let me clarify that I didn't request the feature myself
so I'm not advocating it ;-)
We have a vague proposition in a JIRA ticket, and my intention is to try
give it some objective essence. The team can happily close the ticket with
a "Won't Fix" if we feel that there's no compelling benefit.
That said, the more I think about it, the more I believe Reactive Streams
and Ignite are a valuable combination.
*#1 Compute Jobs*
Reactive Streams would give compute jobs the ability to return partial
results as they are being generated. Basically an API like:
Observable<String> ob = grid.compute().submit(Abc.class,
"foo").toObservable();
would return an Observable to which we can then subscribe to get results as
they are being generated:
ob.subscribe((result) -> System.out.println("Got a result " + result));
The client would get results as they are being emitted by the compute job
server-side.
Of course this is the simplest example. Once we have an Observable we can
do lots and lots of streaming operations on it: merge, drop, folds, etc.
and even combine those operations with other compute jobs. Full set of
operators supported by RxJava:
http://reactivex.io/documentation/operators.html.
So basically a client could send 2, 3, 4... compute jobs and combine their
results together in a streaming fashion:
Observable<String> ob1 = grid.compute().submit(Abc.class,
"foo").toObservable();
Observable<String> ob2 = grid.compute().submit(Def.class,
"foo").toObservable();
Observable<String> ob3 = grid.compute().submit(Ghi.class,
"foo").toObservable();
ob1.flatMap((result) => Observable.just("Observable 1 emitted: " +
result))
.merge(ob2, ob3.skipLast(2))
.subscribe((result) -> System.out.println("Got a result " +
result));
This example would submit 3 jobs, and would combine the streams of all 3
into a single one, but applying a transformation to ob1 and skipping the
last 2 items emitted from ob3.
*#2 Data Streamers*
Similar to above, but for data streamers. Currently our streaming logic is
simple: get a message, transform it and add it to cache. Some streamers
allow ignoring messages, but that's all.
But maybe a user doesn't want to store every single incoming item in the
cache. Maybe they want to store a sliding window result, i.e. receive for 5
seconds, calculate moving averages and store that in the cache.
Or maybe they want to combine multiple data streams together, e.g. combine
3 types of MQTT messages within a window of 10 seconds and only store the
result of an aggregation.
---
I also mentioned messaging and events in my initial email. The rationale
would be similar to the above.
P.S.: For my crude API examples, I'm using RxJava – as that's what I'm
familiar with – not the Reactive Streams API.
P.S. 2: We should put all ideas on a Wiki page – even if we decide to
discard the implementation for now.
Regards,
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk
On Mon, Sep 21, 2015 at 4:49 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> From my standpoint, I would like to see some API examples. How do you
> envision reactive stream support in Ignite at the API level?
>
> D.
>
> On Sun, Sep 20, 2015 at 9:59 PM, Lalit Kumar Jha <la...@gmail.com>
> wrote:
>
> > Yes, lets first explore how reactive-streams can be useful for Ignite.
> >
> > Adding one more candidate to list http://projectreactor.io/docs/, as
> this
> > project:
> > Is for fast-data applications
> > Team are core contributors to the Reactive Streams Specification
> > <https://github.com/reactive-streams/reactive-streams>
> >
> >
> >
> > On Sun, Sep 20, 2015 at 10:33 PM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >
> > wrote:
> >
> > > Guys,
> > >
> > > I think you are digging too deep in implementation possibilities right
> > > away, may be lets start from simpler things?
> > > I don't know much about reactive streams and probably not everyone in
> the
> > > community that educated as well :)
> > >
> > > It would be nice to start with a quick overview of what are they and
> why
> > in
> > > makes sense to
> > > to add such a functionality to Ignite? What real use cases do you see?
> > How
> > > can Ignite
> > > be useful for reactive streams users and vice versa?
> > >
> > > Lets get clear answers to these questions first to avoid writing
> useless
> > > code.
> > >
> > > Sergi
> > >
> > > 2015-09-20 18:55 GMT+03:00 Raul Kripalani <ra...@evosent.com>:
> > >
> > > > Yeah, the idea would be to wrap RxJava 1.0 with the adapter and only
> > use
> > > > the Reactive Streams API where possible from within Ignite.
> > > >
> > > > Nevertheless, the ReactiveX API would also be accessible in case we
> > want
> > > to
> > > > do stuff with Hystrix too.
> > > >
> > > > Raúl.
> > > > On 20 Sep 2015 16:39, "Vishal Garg" <ga...@gmail.com> wrote:
> > > >
> > > > > https://github.com/ReactiveX/RxJava/wiki/R
> > > > > eactive-Streams
> > > > > +1 on your approach ( if we are wrapping 1.0 with adapters??)
> > > > > Raul it looks like there is a path to convergence in RxJava 2.0,
> but
> > > > above
> > > > > link mentions that you could use the standard + integration modules
> > > with
> > > > > wrappers on core 1.0 Apis. Where the modules could be Akka or
> RxJava
> > > > > Vishal
> > > > >
> > > > >
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > > On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <
> > > > > murador.gianfranco@gmail.com> wrote:
> > > > > >
> > > > > > Hy Raul, it could be a valid extension of the Service Grid of
> > Ignite
> > > (
> > > > > if I
> > > > > > understand the point), imho.
> > > > > >
> > > > > > 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> > > > > >
> > > > > >> Hey Igniters,
> > > > > >>
> > > > > >> Lalit Kumar Jha expressed interest [1] in contributing to ticket
> > > > > IGNITE-815
> > > > > >> [2] which aims to bring in Reactive Streams to Ignite.
> > > > > >>
> > > > > >> I wanted to start a brainstorm about which functionalities we'd
> > like
> > > > to
> > > > > >> cover.
> > > > > >>
> > > > > >> Off the top of my head, I can think of merging, filtering,
> > sorting,
> > > > > >> batching, grouping, etc. on Data Streams, Ignite Messages and
> > > Events.
> > > > > Also,
> > > > > >> integration with Continuous Queries would be useful.
> > > > > >>
> > > > > >> Furthermore, I wanted to ask the community if at this point we
> > feel
> > > > > >> comfortable with Reactive Streams per-se or we prefer to go with
> > > > RxJava
> > > > > for
> > > > > >> now. Reactive Streams is a proposed standard whose main Java
> > > > > implementation
> > > > > >> and is slowly gaining traction (e.g. vert.x [3] and MongoDB
> > > > integrations
> > > > > >> [4]), whereas RxJava is an already mature and production-tested
> > > > > >> implementation of the initial ReactiveX design by Microsoft [5]
> > with
> > > > > 7000+
> > > > > >> stars on GitHub.
> > > > > >>
> > > > > >> RxJava is the foundation of Hystrix [6], a powerful library for
> > > > latency
> > > > > and
> > > > > >> fault tolerance management. RxJava does not implement Reactive
> > > Streams
> > > > > as
> > > > > >> of 1.x (as they predated it), but they've vowed support for 2.x
> > [7].
> > > > > >> Moreover, there's already an adapter which passes the TCK [8].
> > > > > >>
> > > > > >> My take is as follows. I'd like to follow the Reactive Streams
> > > > standard,
> > > > > >> but I wouldn't like to lose support for Hystrix (hopefully it'll
> > > > > eventually
> > > > > >> be migrated to Reactive Streams too). So an option is to go with
> > the
> > > > > RxJava
> > > > > >> + adapter design for now to get the best of both worlds. Once
> > RxJava
> > > > > 2.0 is
> > > > > >> out, we can drop the adapter out of the picture.
> > > > > >>
> > > > > >> Thoughts?
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> > > > > >> [2] https://issues.apache.org/jira/browse/IGNITE-815
> > > > > >> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> > > > > >> [4]
> https://mongodb.github.io/mongo-java-driver-reactivestreams/
> > > > > >> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> > > > > >> [6] https://github.com/Netflix/Hystrix
> > > > > >> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> > > > > >> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
> > > > > >>
> > > > > >> Regards,
> > > > > >>
> > > > > >> *Raúl Kripalani*
> > > > > >> Apache Camel PMC Member & Committer | Enterprise Architect, Open
> > > > Source
> > > > > >> Integration specialist
> > > > > >> http://about.me/raulkripalani |
> > > > > http://www.linkedin.com/in/raulkripalani
> > > > > >> http://blog.raulkr.net | twitter: @raulvk
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Dmitriy Setrakyan <ds...@apache.org>.
>From my standpoint, I would like to see some API examples. How do you
envision reactive stream support in Ignite at the API level?
D.
On Sun, Sep 20, 2015 at 9:59 PM, Lalit Kumar Jha <la...@gmail.com>
wrote:
> Yes, lets first explore how reactive-streams can be useful for Ignite.
>
> Adding one more candidate to list http://projectreactor.io/docs/, as this
> project:
> Is for fast-data applications
> Team are core contributors to the Reactive Streams Specification
> <https://github.com/reactive-streams/reactive-streams>
>
>
>
> On Sun, Sep 20, 2015 at 10:33 PM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> wrote:
>
> > Guys,
> >
> > I think you are digging too deep in implementation possibilities right
> > away, may be lets start from simpler things?
> > I don't know much about reactive streams and probably not everyone in the
> > community that educated as well :)
> >
> > It would be nice to start with a quick overview of what are they and why
> in
> > makes sense to
> > to add such a functionality to Ignite? What real use cases do you see?
> How
> > can Ignite
> > be useful for reactive streams users and vice versa?
> >
> > Lets get clear answers to these questions first to avoid writing useless
> > code.
> >
> > Sergi
> >
> > 2015-09-20 18:55 GMT+03:00 Raul Kripalani <ra...@evosent.com>:
> >
> > > Yeah, the idea would be to wrap RxJava 1.0 with the adapter and only
> use
> > > the Reactive Streams API where possible from within Ignite.
> > >
> > > Nevertheless, the ReactiveX API would also be accessible in case we
> want
> > to
> > > do stuff with Hystrix too.
> > >
> > > Raúl.
> > > On 20 Sep 2015 16:39, "Vishal Garg" <ga...@gmail.com> wrote:
> > >
> > > > https://github.com/ReactiveX/RxJava/wiki/R
> > > > eactive-Streams
> > > > +1 on your approach ( if we are wrapping 1.0 with adapters??)
> > > > Raul it looks like there is a path to convergence in RxJava 2.0, but
> > > above
> > > > link mentions that you could use the standard + integration modules
> > with
> > > > wrappers on core 1.0 Apis. Where the modules could be Akka or RxJava
> > > > Vishal
> > > >
> > > >
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <
> > > > murador.gianfranco@gmail.com> wrote:
> > > > >
> > > > > Hy Raul, it could be a valid extension of the Service Grid of
> Ignite
> > (
> > > > if I
> > > > > understand the point), imho.
> > > > >
> > > > > 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> > > > >
> > > > >> Hey Igniters,
> > > > >>
> > > > >> Lalit Kumar Jha expressed interest [1] in contributing to ticket
> > > > IGNITE-815
> > > > >> [2] which aims to bring in Reactive Streams to Ignite.
> > > > >>
> > > > >> I wanted to start a brainstorm about which functionalities we'd
> like
> > > to
> > > > >> cover.
> > > > >>
> > > > >> Off the top of my head, I can think of merging, filtering,
> sorting,
> > > > >> batching, grouping, etc. on Data Streams, Ignite Messages and
> > Events.
> > > > Also,
> > > > >> integration with Continuous Queries would be useful.
> > > > >>
> > > > >> Furthermore, I wanted to ask the community if at this point we
> feel
> > > > >> comfortable with Reactive Streams per-se or we prefer to go with
> > > RxJava
> > > > for
> > > > >> now. Reactive Streams is a proposed standard whose main Java
> > > > implementation
> > > > >> and is slowly gaining traction (e.g. vert.x [3] and MongoDB
> > > integrations
> > > > >> [4]), whereas RxJava is an already mature and production-tested
> > > > >> implementation of the initial ReactiveX design by Microsoft [5]
> with
> > > > 7000+
> > > > >> stars on GitHub.
> > > > >>
> > > > >> RxJava is the foundation of Hystrix [6], a powerful library for
> > > latency
> > > > and
> > > > >> fault tolerance management. RxJava does not implement Reactive
> > Streams
> > > > as
> > > > >> of 1.x (as they predated it), but they've vowed support for 2.x
> [7].
> > > > >> Moreover, there's already an adapter which passes the TCK [8].
> > > > >>
> > > > >> My take is as follows. I'd like to follow the Reactive Streams
> > > standard,
> > > > >> but I wouldn't like to lose support for Hystrix (hopefully it'll
> > > > eventually
> > > > >> be migrated to Reactive Streams too). So an option is to go with
> the
> > > > RxJava
> > > > >> + adapter design for now to get the best of both worlds. Once
> RxJava
> > > > 2.0 is
> > > > >> out, we can drop the adapter out of the picture.
> > > > >>
> > > > >> Thoughts?
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> > > > >> [2] https://issues.apache.org/jira/browse/IGNITE-815
> > > > >> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> > > > >> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
> > > > >> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> > > > >> [6] https://github.com/Netflix/Hystrix
> > > > >> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> > > > >> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
> > > > >>
> > > > >> Regards,
> > > > >>
> > > > >> *Raúl Kripalani*
> > > > >> Apache Camel PMC Member & Committer | Enterprise Architect, Open
> > > Source
> > > > >> Integration specialist
> > > > >> http://about.me/raulkripalani |
> > > > http://www.linkedin.com/in/raulkripalani
> > > > >> http://blog.raulkr.net | twitter: @raulvk
> > > > >>
> > > >
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Lalit Kumar Jha <la...@gmail.com>.
Yes, lets first explore how reactive-streams can be useful for Ignite.
Adding one more candidate to list http://projectreactor.io/docs/, as this
project:
Is for fast-data applications
Team are core contributors to the Reactive Streams Specification
<https://github.com/reactive-streams/reactive-streams>
On Sun, Sep 20, 2015 at 10:33 PM, Sergi Vladykin <se...@gmail.com>
wrote:
> Guys,
>
> I think you are digging too deep in implementation possibilities right
> away, may be lets start from simpler things?
> I don't know much about reactive streams and probably not everyone in the
> community that educated as well :)
>
> It would be nice to start with a quick overview of what are they and why in
> makes sense to
> to add such a functionality to Ignite? What real use cases do you see? How
> can Ignite
> be useful for reactive streams users and vice versa?
>
> Lets get clear answers to these questions first to avoid writing useless
> code.
>
> Sergi
>
> 2015-09-20 18:55 GMT+03:00 Raul Kripalani <ra...@evosent.com>:
>
> > Yeah, the idea would be to wrap RxJava 1.0 with the adapter and only use
> > the Reactive Streams API where possible from within Ignite.
> >
> > Nevertheless, the ReactiveX API would also be accessible in case we want
> to
> > do stuff with Hystrix too.
> >
> > Raúl.
> > On 20 Sep 2015 16:39, "Vishal Garg" <ga...@gmail.com> wrote:
> >
> > > https://github.com/ReactiveX/RxJava/wiki/R
> > > eactive-Streams
> > > +1 on your approach ( if we are wrapping 1.0 with adapters??)
> > > Raul it looks like there is a path to convergence in RxJava 2.0, but
> > above
> > > link mentions that you could use the standard + integration modules
> with
> > > wrappers on core 1.0 Apis. Where the modules could be Akka or RxJava
> > > Vishal
> > >
> > >
> > >
> > > Sent from my iPhone
> > >
> > > > On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <
> > > murador.gianfranco@gmail.com> wrote:
> > > >
> > > > Hy Raul, it could be a valid extension of the Service Grid of Ignite
> (
> > > if I
> > > > understand the point), imho.
> > > >
> > > > 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> > > >
> > > >> Hey Igniters,
> > > >>
> > > >> Lalit Kumar Jha expressed interest [1] in contributing to ticket
> > > IGNITE-815
> > > >> [2] which aims to bring in Reactive Streams to Ignite.
> > > >>
> > > >> I wanted to start a brainstorm about which functionalities we'd like
> > to
> > > >> cover.
> > > >>
> > > >> Off the top of my head, I can think of merging, filtering, sorting,
> > > >> batching, grouping, etc. on Data Streams, Ignite Messages and
> Events.
> > > Also,
> > > >> integration with Continuous Queries would be useful.
> > > >>
> > > >> Furthermore, I wanted to ask the community if at this point we feel
> > > >> comfortable with Reactive Streams per-se or we prefer to go with
> > RxJava
> > > for
> > > >> now. Reactive Streams is a proposed standard whose main Java
> > > implementation
> > > >> and is slowly gaining traction (e.g. vert.x [3] and MongoDB
> > integrations
> > > >> [4]), whereas RxJava is an already mature and production-tested
> > > >> implementation of the initial ReactiveX design by Microsoft [5] with
> > > 7000+
> > > >> stars on GitHub.
> > > >>
> > > >> RxJava is the foundation of Hystrix [6], a powerful library for
> > latency
> > > and
> > > >> fault tolerance management. RxJava does not implement Reactive
> Streams
> > > as
> > > >> of 1.x (as they predated it), but they've vowed support for 2.x [7].
> > > >> Moreover, there's already an adapter which passes the TCK [8].
> > > >>
> > > >> My take is as follows. I'd like to follow the Reactive Streams
> > standard,
> > > >> but I wouldn't like to lose support for Hystrix (hopefully it'll
> > > eventually
> > > >> be migrated to Reactive Streams too). So an option is to go with the
> > > RxJava
> > > >> + adapter design for now to get the best of both worlds. Once RxJava
> > > 2.0 is
> > > >> out, we can drop the adapter out of the picture.
> > > >>
> > > >> Thoughts?
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> > > >> [2] https://issues.apache.org/jira/browse/IGNITE-815
> > > >> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> > > >> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
> > > >> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> > > >> [6] https://github.com/Netflix/Hystrix
> > > >> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> > > >> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
> > > >>
> > > >> Regards,
> > > >>
> > > >> *Raúl Kripalani*
> > > >> Apache Camel PMC Member & Committer | Enterprise Architect, Open
> > Source
> > > >> Integration specialist
> > > >> http://about.me/raulkripalani |
> > > http://www.linkedin.com/in/raulkripalani
> > > >> http://blog.raulkr.net | twitter: @raulvk
> > > >>
> > >
> >
>
Re: Brainstorming about Reactive Streams
Posted by Sergi Vladykin <se...@gmail.com>.
Guys,
I think you are digging too deep in implementation possibilities right
away, may be lets start from simpler things?
I don't know much about reactive streams and probably not everyone in the
community that educated as well :)
It would be nice to start with a quick overview of what are they and why in
makes sense to
to add such a functionality to Ignite? What real use cases do you see? How
can Ignite
be useful for reactive streams users and vice versa?
Lets get clear answers to these questions first to avoid writing useless
code.
Sergi
2015-09-20 18:55 GMT+03:00 Raul Kripalani <ra...@evosent.com>:
> Yeah, the idea would be to wrap RxJava 1.0 with the adapter and only use
> the Reactive Streams API where possible from within Ignite.
>
> Nevertheless, the ReactiveX API would also be accessible in case we want to
> do stuff with Hystrix too.
>
> Raúl.
> On 20 Sep 2015 16:39, "Vishal Garg" <ga...@gmail.com> wrote:
>
> > https://github.com/ReactiveX/RxJava/wiki/R
> > eactive-Streams
> > +1 on your approach ( if we are wrapping 1.0 with adapters??)
> > Raul it looks like there is a path to convergence in RxJava 2.0, but
> above
> > link mentions that you could use the standard + integration modules with
> > wrappers on core 1.0 Apis. Where the modules could be Akka or RxJava
> > Vishal
> >
> >
> >
> > Sent from my iPhone
> >
> > > On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <
> > murador.gianfranco@gmail.com> wrote:
> > >
> > > Hy Raul, it could be a valid extension of the Service Grid of Ignite (
> > if I
> > > understand the point), imho.
> > >
> > > 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> > >
> > >> Hey Igniters,
> > >>
> > >> Lalit Kumar Jha expressed interest [1] in contributing to ticket
> > IGNITE-815
> > >> [2] which aims to bring in Reactive Streams to Ignite.
> > >>
> > >> I wanted to start a brainstorm about which functionalities we'd like
> to
> > >> cover.
> > >>
> > >> Off the top of my head, I can think of merging, filtering, sorting,
> > >> batching, grouping, etc. on Data Streams, Ignite Messages and Events.
> > Also,
> > >> integration with Continuous Queries would be useful.
> > >>
> > >> Furthermore, I wanted to ask the community if at this point we feel
> > >> comfortable with Reactive Streams per-se or we prefer to go with
> RxJava
> > for
> > >> now. Reactive Streams is a proposed standard whose main Java
> > implementation
> > >> and is slowly gaining traction (e.g. vert.x [3] and MongoDB
> integrations
> > >> [4]), whereas RxJava is an already mature and production-tested
> > >> implementation of the initial ReactiveX design by Microsoft [5] with
> > 7000+
> > >> stars on GitHub.
> > >>
> > >> RxJava is the foundation of Hystrix [6], a powerful library for
> latency
> > and
> > >> fault tolerance management. RxJava does not implement Reactive Streams
> > as
> > >> of 1.x (as they predated it), but they've vowed support for 2.x [7].
> > >> Moreover, there's already an adapter which passes the TCK [8].
> > >>
> > >> My take is as follows. I'd like to follow the Reactive Streams
> standard,
> > >> but I wouldn't like to lose support for Hystrix (hopefully it'll
> > eventually
> > >> be migrated to Reactive Streams too). So an option is to go with the
> > RxJava
> > >> + adapter design for now to get the best of both worlds. Once RxJava
> > 2.0 is
> > >> out, we can drop the adapter out of the picture.
> > >>
> > >> Thoughts?
> > >>
> > >> [1]
> > >>
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> > >> [2] https://issues.apache.org/jira/browse/IGNITE-815
> > >> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> > >> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
> > >> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> > >> [6] https://github.com/Netflix/Hystrix
> > >> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> > >> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
> > >>
> > >> Regards,
> > >>
> > >> *Raúl Kripalani*
> > >> Apache Camel PMC Member & Committer | Enterprise Architect, Open
> Source
> > >> Integration specialist
> > >> http://about.me/raulkripalani |
> > http://www.linkedin.com/in/raulkripalani
> > >> http://blog.raulkr.net | twitter: @raulvk
> > >>
> >
>
Re: Brainstorming about Reactive Streams
Posted by Raul Kripalani <ra...@evosent.com>.
Yeah, the idea would be to wrap RxJava 1.0 with the adapter and only use
the Reactive Streams API where possible from within Ignite.
Nevertheless, the ReactiveX API would also be accessible in case we want to
do stuff with Hystrix too.
Raúl.
On 20 Sep 2015 16:39, "Vishal Garg" <ga...@gmail.com> wrote:
> https://github.com/ReactiveX/RxJava/wiki/R
> eactive-Streams
> +1 on your approach ( if we are wrapping 1.0 with adapters??)
> Raul it looks like there is a path to convergence in RxJava 2.0, but above
> link mentions that you could use the standard + integration modules with
> wrappers on core 1.0 Apis. Where the modules could be Akka or RxJava
> Vishal
>
>
>
> Sent from my iPhone
>
> > On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <
> murador.gianfranco@gmail.com> wrote:
> >
> > Hy Raul, it could be a valid extension of the Service Grid of Ignite (
> if I
> > understand the point), imho.
> >
> > 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> >
> >> Hey Igniters,
> >>
> >> Lalit Kumar Jha expressed interest [1] in contributing to ticket
> IGNITE-815
> >> [2] which aims to bring in Reactive Streams to Ignite.
> >>
> >> I wanted to start a brainstorm about which functionalities we'd like to
> >> cover.
> >>
> >> Off the top of my head, I can think of merging, filtering, sorting,
> >> batching, grouping, etc. on Data Streams, Ignite Messages and Events.
> Also,
> >> integration with Continuous Queries would be useful.
> >>
> >> Furthermore, I wanted to ask the community if at this point we feel
> >> comfortable with Reactive Streams per-se or we prefer to go with RxJava
> for
> >> now. Reactive Streams is a proposed standard whose main Java
> implementation
> >> and is slowly gaining traction (e.g. vert.x [3] and MongoDB integrations
> >> [4]), whereas RxJava is an already mature and production-tested
> >> implementation of the initial ReactiveX design by Microsoft [5] with
> 7000+
> >> stars on GitHub.
> >>
> >> RxJava is the foundation of Hystrix [6], a powerful library for latency
> and
> >> fault tolerance management. RxJava does not implement Reactive Streams
> as
> >> of 1.x (as they predated it), but they've vowed support for 2.x [7].
> >> Moreover, there's already an adapter which passes the TCK [8].
> >>
> >> My take is as follows. I'd like to follow the Reactive Streams standard,
> >> but I wouldn't like to lose support for Hystrix (hopefully it'll
> eventually
> >> be migrated to Reactive Streams too). So an option is to go with the
> RxJava
> >> + adapter design for now to get the best of both worlds. Once RxJava
> 2.0 is
> >> out, we can drop the adapter out of the picture.
> >>
> >> Thoughts?
> >>
> >> [1]
> >>
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> >> [2] https://issues.apache.org/jira/browse/IGNITE-815
> >> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> >> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
> >> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> >> [6] https://github.com/Netflix/Hystrix
> >> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> >> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
> >>
> >> Regards,
> >>
> >> *Raúl Kripalani*
> >> Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
> >> Integration specialist
> >> http://about.me/raulkripalani |
> http://www.linkedin.com/in/raulkripalani
> >> http://blog.raulkr.net | twitter: @raulvk
> >>
>
Re: Brainstorming about Reactive Streams
Posted by Vishal Garg <ga...@gmail.com>.
https://github.com/ReactiveX/RxJava/wiki/R
eactive-Streams
+1 on your approach ( if we are wrapping 1.0 with adapters??)
Raul it looks like there is a path to convergence in RxJava 2.0, but above link mentions that you could use the standard + integration modules with wrappers on core 1.0 Apis. Where the modules could be Akka or RxJava
Vishal
Sent from my iPhone
> On Sep 20, 2015, at 7:03 AM, Gianfranco Murador <mu...@gmail.com> wrote:
>
> Hy Raul, it could be a valid extension of the Service Grid of Ignite ( if I
> understand the point), imho.
>
> 2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
>
>> Hey Igniters,
>>
>> Lalit Kumar Jha expressed interest [1] in contributing to ticket IGNITE-815
>> [2] which aims to bring in Reactive Streams to Ignite.
>>
>> I wanted to start a brainstorm about which functionalities we'd like to
>> cover.
>>
>> Off the top of my head, I can think of merging, filtering, sorting,
>> batching, grouping, etc. on Data Streams, Ignite Messages and Events. Also,
>> integration with Continuous Queries would be useful.
>>
>> Furthermore, I wanted to ask the community if at this point we feel
>> comfortable with Reactive Streams per-se or we prefer to go with RxJava for
>> now. Reactive Streams is a proposed standard whose main Java implementation
>> and is slowly gaining traction (e.g. vert.x [3] and MongoDB integrations
>> [4]), whereas RxJava is an already mature and production-tested
>> implementation of the initial ReactiveX design by Microsoft [5] with 7000+
>> stars on GitHub.
>>
>> RxJava is the foundation of Hystrix [6], a powerful library for latency and
>> fault tolerance management. RxJava does not implement Reactive Streams as
>> of 1.x (as they predated it), but they've vowed support for 2.x [7].
>> Moreover, there's already an adapter which passes the TCK [8].
>>
>> My take is as follows. I'd like to follow the Reactive Streams standard,
>> but I wouldn't like to lose support for Hystrix (hopefully it'll eventually
>> be migrated to Reactive Streams too). So an option is to go with the RxJava
>> + adapter design for now to get the best of both worlds. Once RxJava 2.0 is
>> out, we can drop the adapter out of the picture.
>>
>> Thoughts?
>>
>> [1]
>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
>> [2] https://issues.apache.org/jira/browse/IGNITE-815
>> [3] http://vertx.io/docs/vertx-reactive-streams/java/
>> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
>> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
>> [6] https://github.com/Netflix/Hystrix
>> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
>> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
>>
>> Regards,
>>
>> *Raúl Kripalani*
>> Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
>> Integration specialist
>> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
>> http://blog.raulkr.net | twitter: @raulvk
>>
Re: Brainstorming about Reactive Streams
Posted by Gianfranco Murador <mu...@gmail.com>.
Hy Raul, it could be a valid extension of the Service Grid of Ignite ( if I
understand the point), imho.
2015-09-20 14:37 GMT+02:00 Raul Kripalani <ra...@apache.org>:
> Hey Igniters,
>
> Lalit Kumar Jha expressed interest [1] in contributing to ticket IGNITE-815
> [2] which aims to bring in Reactive Streams to Ignite.
>
> I wanted to start a brainstorm about which functionalities we'd like to
> cover.
>
> Off the top of my head, I can think of merging, filtering, sorting,
> batching, grouping, etc. on Data Streams, Ignite Messages and Events. Also,
> integration with Continuous Queries would be useful.
>
> Furthermore, I wanted to ask the community if at this point we feel
> comfortable with Reactive Streams per-se or we prefer to go with RxJava for
> now. Reactive Streams is a proposed standard whose main Java implementation
> and is slowly gaining traction (e.g. vert.x [3] and MongoDB integrations
> [4]), whereas RxJava is an already mature and production-tested
> implementation of the initial ReactiveX design by Microsoft [5] with 7000+
> stars on GitHub.
>
> RxJava is the foundation of Hystrix [6], a powerful library for latency and
> fault tolerance management. RxJava does not implement Reactive Streams as
> of 1.x (as they predated it), but they've vowed support for 2.x [7].
> Moreover, there's already an adapter which passes the TCK [8].
>
> My take is as follows. I'd like to follow the Reactive Streams standard,
> but I wouldn't like to lose support for Hystrix (hopefully it'll eventually
> be migrated to Reactive Streams too). So an option is to go with the RxJava
> + adapter design for now to get the best of both worlds. Once RxJava 2.0 is
> out, we can drop the adapter out of the picture.
>
> Thoughts?
>
> [1]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Hello-Apache-Ignite-Dev-tp3137p3315.html
> [2] https://issues.apache.org/jira/browse/IGNITE-815
> [3] http://vertx.io/docs/vertx-reactive-streams/java/
> [4] https://mongodb.github.io/mongo-java-driver-reactivestreams/
> [5] https://msdn.microsoft.com/en-us/data/gg577609.aspx
> [6] https://github.com/Netflix/Hystrix
> [7] https://github.com/ReactiveX/RxJava/wiki/Reactive-Streams
> [8] https://github.com/ReactiveX/RxJavaReactiveStreams
>
> Regards,
>
> *Raúl Kripalani*
> Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
> Integration specialist
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>