You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@karaf.apache.org by Christian Schneider <ch...@die-schneider.net> on 2017/07/19 11:02:33 UTC
[Proposal] New subproject rcomp - Reactive components framework
Scope
I recently experimented with reactive streams and built a small
component framework on top of this spec.
See https://github.com/cschneider/reactive-components
The goal is to have a small API that can encapsulate a protocol and
transport. The code using such a reactive component should not directly
depend on the specifics of the transport or protocol. Another goal is to
have reactive features like back pressure. Ultimately I am searching for
something like Apache Camel Components but with a lot less coupling. In
camel the big problem is that components depend on camel core which
unfortunately is much more than a component API. So any camel component
is coupled quite tightly to all of camel core.
Proposal
I propose to donate my code to Apache and establish this as a Apache
Karaf sub project. Some people like Jean-Baptiste and Hadrian have
already expressed that they support this and I hope for some more
feedback and help.
I chose the Karaf project at the moment but am also open to placing this
in another Apache project. Some matching projects would be Apache Camel,
Aries or Felix.
Component API
I was trying to find the simplest API that would allow similar
components to camel in one way mode.
public interface RComponent {
<T> Publisher<T> from(String destination, Class<T> type);
<T> Subscriber<T> to(String destination, Class<T> type);
}
A component is a factory for Publishers and Subscribers. From and to
have similar meaning as in camel. The component can be given a source /
target type to produce / consume. So with the OSGi Converter spec this
would allow to have type safe messaging without coding the conversion in
every component. Each component is exposed as a service which
encapsulates most of the configuration. All endpoint specific
configuration can be done using the destination String.
Publisher and Subscriber are interfaces from the reactive streams api
(http://www.reactive-streams.org/). So they are well defined and have
zero additional dependencies.
I also considered to use OSGi push streams which is an OSGi spec and
would also be an interesting foundation. I decided against that though
as push streams have no API that is separate from the DSL and will
probably not be used a lot outside of OSGi.
See the examples for how to use this in practice.
https://github.com/cschneider/reactive-components
Possible use cases
Two big use cases are reactive microservices that need messaging as well
as plain camel like integrations.
Another case are the Apache Karaf decanter collectors and appenders.
Currently they use a decanter specific API but they could easily be
converted into the more general rcomp api.
We could also create a bridge to camel components to leverage the many
existing camel components using the rcomp API as well as offering rcomp
components to camel.
Components alone are of course not enough. One big strength of Apache
Camel is the DSL. In case of rcomp I propose to not create our own DSL
and instead use existing DSLs that work well in OSGi. Two examples:
Akka and reactive streams
https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
Reactor and reactive streams
https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring
Another integration is with REST. It is already possible to integrate
CXF Rest services with reactive streams using some adapters but we could
have native integration.
Risks and Opportunities
The main risk I see is not gathering a critical mass of components to
draw more people.
Another risk is that the RComponent API or the reactor streams have some
unexpected limitations.
The big opportunity I see is that the rcomp API is very simple so the
barrier of entry is low.
I also hope that this might become a new foundation for a simpler and
more modern Apache Camel.
So this all depends on getting some support by you all.
Christian
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Matt Sicker <bo...@gmail.com>.
I definitely like the idea. Camel-rx is for RxJava 1.x which is not based
on the reactive streams API (RxJava 2.x, however, is based on that API).
On 19 July 2017 at 06:51, Guillaume Nodet <gn...@apache.org> wrote:
> At first glance, it looks similar related to http://camel.apache.org/rx.
> html
> Also, given what you say about Camel and OSGi Push Streams, I think it
> would make more sense to discuss that inside the Camel project instead.
>
> 2017-07-19 13:02 GMT+02:00 Christian Schneider <ch...@die-schneider.net>:
>
> >
> > Scope
> >
> > I recently experimented with reactive streams and built a small component
> > framework on top of this spec.
> > See https://github.com/cschneider/reactive-components
> >
> > The goal is to have a small API that can encapsulate a protocol and
> > transport. The code using such a reactive component should not directly
> > depend on the specifics of the transport or protocol. Another goal is to
> > have reactive features like back pressure. Ultimately I am searching for
> > something like Apache Camel Components but with a lot less coupling. In
> > camel the big problem is that components depend on camel core which
> > unfortunately is much more than a component API. So any camel component
> is
> > coupled quite tightly to all of camel core.
> >
> >
> > Proposal
> >
> > I propose to donate my code to Apache and establish this as a Apache
> Karaf
> > sub project. Some people like Jean-Baptiste and Hadrian have already
> > expressed that they support this and I hope for some more feedback and
> help.
> >
> > I chose the Karaf project at the moment but am also open to placing this
> > in another Apache project. Some matching projects would be Apache Camel,
> > Aries or Felix.
> >
> >
> > Component API
> >
> > I was trying to find the simplest API that would allow similar components
> > to camel in one way mode.
> >
> > public interface RComponent {
> > <T> Publisher<T> from(String destination, Class<T> type);
> > <T> Subscriber<T> to(String destination, Class<T> type);
> > }
> >
> > A component is a factory for Publishers and Subscribers. From and to have
> > similar meaning as in camel. The component can be given a source / target
> > type to produce / consume. So with the OSGi Converter spec this would
> allow
> > to have type safe messaging without coding the conversion in every
> > component. Each component is exposed as a service which encapsulates most
> > of the configuration. All endpoint specific configuration can be done
> using
> > the destination String.
> >
> > Publisher and Subscriber are interfaces from the reactive streams api (
> > http://www.reactive-streams.org/). So they are well defined and have
> zero
> > additional dependencies.
> >
> > I also considered to use OSGi push streams which is an OSGi spec and
> would
> > also be an interesting foundation. I decided against that though as push
> > streams have no API that is separate from the DSL and will probably not
> be
> > used a lot outside of OSGi.
> >
> > See the examples for how to use this in practice.
> > https://github.com/cschneider/reactive-components
> >
> >
> > Possible use cases
> >
> > Two big use cases are reactive microservices that need messaging as well
> > as plain camel like integrations.
> > Another case are the Apache Karaf decanter collectors and appenders.
> > Currently they use a decanter specific API but they could easily be
> > converted into the more general rcomp api.
> > We could also create a bridge to camel components to leverage the many
> > existing camel components using the rcomp API as well as offering rcomp
> > components to camel.
> >
> > Components alone are of course not enough. One big strength of Apache
> > Camel is the DSL. In case of rcomp I propose to not create our own DSL
> and
> > instead use existing DSLs that work well in OSGi. Two examples:
> >
> > Akka and reactive streams
> > https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
> >
> > Reactor and reactive streams
> > https://de.slideshare.net/StphaneMaldini/reactor-30-a-reacti
> > ve-foundation-for-java-8-and-spring
> >
> > Another integration is with REST. It is already possible to integrate CXF
> > Rest services with reactive streams using some adapters but we could have
> > native integration.
> >
> >
> > Risks and Opportunities
> >
> > The main risk I see is not gathering a critical mass of components to
> draw
> > more people.
> > Another risk is that the RComponent API or the reactor streams have some
> > unexpected limitations.
> > The big opportunity I see is that the rcomp API is very simple so the
> > barrier of entry is low.
> > I also hope that this might become a new foundation for a simpler and
> more
> > modern Apache Camel.
> >
> > So this all depends on getting some support by you all.
> >
> > Christian
> >
> >
> > --
> > Christian Schneider
> > http://www.liquid-reality.de
> >
> > Open Source Architect
> > http://www.talend.com
> >
> >
>
>
> --
> ------------------------
> Guillaume Nodet
>
--
Matt Sicker <bo...@gmail.com>
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Guillaume Nodet <gn...@apache.org>.
At first glance, it looks similar related to http://camel.apache.org/rx.html
Also, given what you say about Camel and OSGi Push Streams, I think it
would make more sense to discuss that inside the Camel project instead.
2017-07-19 13:02 GMT+02:00 Christian Schneider <ch...@die-schneider.net>:
>
> Scope
>
> I recently experimented with reactive streams and built a small component
> framework on top of this spec.
> See https://github.com/cschneider/reactive-components
>
> The goal is to have a small API that can encapsulate a protocol and
> transport. The code using such a reactive component should not directly
> depend on the specifics of the transport or protocol. Another goal is to
> have reactive features like back pressure. Ultimately I am searching for
> something like Apache Camel Components but with a lot less coupling. In
> camel the big problem is that components depend on camel core which
> unfortunately is much more than a component API. So any camel component is
> coupled quite tightly to all of camel core.
>
>
> Proposal
>
> I propose to donate my code to Apache and establish this as a Apache Karaf
> sub project. Some people like Jean-Baptiste and Hadrian have already
> expressed that they support this and I hope for some more feedback and help.
>
> I chose the Karaf project at the moment but am also open to placing this
> in another Apache project. Some matching projects would be Apache Camel,
> Aries or Felix.
>
>
> Component API
>
> I was trying to find the simplest API that would allow similar components
> to camel in one way mode.
>
> public interface RComponent {
> <T> Publisher<T> from(String destination, Class<T> type);
> <T> Subscriber<T> to(String destination, Class<T> type);
> }
>
> A component is a factory for Publishers and Subscribers. From and to have
> similar meaning as in camel. The component can be given a source / target
> type to produce / consume. So with the OSGi Converter spec this would allow
> to have type safe messaging without coding the conversion in every
> component. Each component is exposed as a service which encapsulates most
> of the configuration. All endpoint specific configuration can be done using
> the destination String.
>
> Publisher and Subscriber are interfaces from the reactive streams api (
> http://www.reactive-streams.org/). So they are well defined and have zero
> additional dependencies.
>
> I also considered to use OSGi push streams which is an OSGi spec and would
> also be an interesting foundation. I decided against that though as push
> streams have no API that is separate from the DSL and will probably not be
> used a lot outside of OSGi.
>
> See the examples for how to use this in practice.
> https://github.com/cschneider/reactive-components
>
>
> Possible use cases
>
> Two big use cases are reactive microservices that need messaging as well
> as plain camel like integrations.
> Another case are the Apache Karaf decanter collectors and appenders.
> Currently they use a decanter specific API but they could easily be
> converted into the more general rcomp api.
> We could also create a bridge to camel components to leverage the many
> existing camel components using the rcomp API as well as offering rcomp
> components to camel.
>
> Components alone are of course not enough. One big strength of Apache
> Camel is the DSL. In case of rcomp I propose to not create our own DSL and
> instead use existing DSLs that work well in OSGi. Two examples:
>
> Akka and reactive streams
> https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
>
> Reactor and reactive streams
> https://de.slideshare.net/StphaneMaldini/reactor-30-a-reacti
> ve-foundation-for-java-8-and-spring
>
> Another integration is with REST. It is already possible to integrate CXF
> Rest services with reactive streams using some adapters but we could have
> native integration.
>
>
> Risks and Opportunities
>
> The main risk I see is not gathering a critical mass of components to draw
> more people.
> Another risk is that the RComponent API or the reactor streams have some
> unexpected limitations.
> The big opportunity I see is that the rcomp API is very simple so the
> barrier of entry is low.
> I also hope that this might become a new foundation for a simpler and more
> modern Apache Camel.
>
> So this all depends on getting some support by you all.
>
> Christian
>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>
--
------------------------
Guillaume Nodet
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
fair enough, I would have wait for a formal vote before taking the initiative.
We had a proposal/discussion thread, but no formal vote for the acceptance.
I think in term of legal we are good as it's under the Apache license already.
I will do a full review later this week.
Regards
JB
On 10/22/2017 06:16 PM, Christian Schneider wrote:
> I have now organized the repo and did the initial push with my current code of
> reactive components.
>
> You can find the repo at:
> https://github.com/apache/karaf-reactive-components
>
> I would still be happy about a review. I think we should have no licensing
> issues as the code was only developed by me and I have an ICLA.
> So I can confirm that this is fully apache licensed and written by me.
>
> Please let me know if we need something more formal.
>
> Best
> Christian
>
> 2017-08-08 14:26 GMT+02:00 Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>>:
>
> Hi
>
> I will do a new review round tomorrow morning.
>
> Thanks
> Regards
> JB
>
> On Aug 8, 2017, 13:04, at 13:04, Christian Schneider
> <chris@die-schneider.net <ma...@die-schneider.net>> wrote:
> >I now adapted the package names as well as the maven coordinates. All
> >files should now also have the apache headers.
> >I also made sure the build now works without any running mqtt or kafka
> >server.
> >
> >I would be happy about a quick review of the current status. If there
> >are no objections then I will go ahead and ask for a git repository.
> >
> >Christian
> >
> >On 03.08.2017 08:02, Jean-Baptiste Onofré wrote:
> >> Hi Christian,
> >>
> >> the proposal is interesting, and I see a first potential use in
> >> Decanter for sure.
> >>
> >> The comment about Camel is fair as it looks very similar at first
> >glance.
> >>
> >> However, the scope is slightly different IMHO.
> >>
> >> I checked about the legal. The code should be cleanup for donation
> >> (ASF headers, package names, ...). About the dependency license, for
> >> the reactive-streams, it' OK as it uses Creative Commons Zero into
> >the
> >> Public Domain. And Reactor is already under Apache license.
> >>
> >> The DSL is a hot topic. I understand that leveraging Akka or Reactor
> >> is easier, but I'm a bit concern that we could be too much "coupled".
> >> Maybe our own simple DSL could help. Thoughts ?
> >>
> >> Thanks anyway !
> >> Regards
> >> JB
> >>
> >> On 07/19/2017 01:02 PM, Christian Schneider wrote:
> >>>
> >>> Scope
> >>>
> >>> I recently experimented with reactive streams and built a small
> >>> component framework on top of this spec.
> >>> See https://github.com/cschneider/reactive-components
> <https://github.com/cschneider/reactive-components>
> >>>
> >>> The goal is to have a small API that can encapsulate a protocol and
> >>> transport. The code using such a reactive component should not
> >>> directly depend on the specifics of the transport or protocol.
> >>> Another goal is to have reactive features like back pressure.
> >>> Ultimately I am searching for something like Apache Camel Components
> >
> >>> but with a lot less coupling. In camel the big problem is that
> >>> components depend on camel core which unfortunately is much more
> >than
> >>> a component API. So any camel component is coupled quite tightly to
> >>> all of camel core.
> >>>
> >>>
> >>> Proposal
> >>>
> >>> I propose to donate my code to Apache and establish this as a Apache
> >
> >>> Karaf sub project. Some people like Jean-Baptiste and Hadrian have
> >>> already expressed that they support this and I hope for some more
> >>> feedback and help.
> >>>
> >>> I chose the Karaf project at the moment but am also open to placing
> >>> this in another Apache project. Some matching projects would be
> >>> Apache Camel, Aries or Felix.
> >>>
> >>>
> >>> Component API
> >>>
> >>> I was trying to find the simplest API that would allow similar
> >>> components to camel in one way mode.
> >>>
> >>> public interface RComponent {
> >>> <T> Publisher<T> from(String destination, Class<T> type);
> >>> <T> Subscriber<T> to(String destination, Class<T> type);
> >>> }
> >>>
> >>> A component is a factory for Publishers and Subscribers. From and to
> >
> >>> have similar meaning as in camel. The component can be given a
> >source
> >>> / target type to produce / consume. So with the OSGi Converter spec
> >>> this would allow to have type safe messaging without coding the
> >>> conversion in every component. Each component is exposed as a
> >service
> >>> which encapsulates most of the configuration. All endpoint specific
> >>> configuration can be done using the destination String.
> >>>
> >>> Publisher and Subscriber are interfaces from the reactive streams
> >api
> >>> (http://www.reactive-streams.org/ <http://www.reactive-streams.org/>).
> So they are well defined and
> >have
> >>> zero additional dependencies.
> >>>
> >>> I also considered to use OSGi push streams which is an OSGi spec and
> >
> >>> would also be an interesting foundation. I decided against that
> >>> though as push streams have no API that is separate from the DSL and
> >
> >>> will probably not be used a lot outside of OSGi.
> >>>
> >>> See the examples for how to use this in practice.
> >>> https://github.com/cschneider/reactive-components
> <https://github.com/cschneider/reactive-components>
> >>>
> >>>
> >>> Possible use cases
> >>>
> >>> Two big use cases are reactive microservices that need messaging as
> >>> well as plain camel like integrations.
> >>> Another case are the Apache Karaf decanter collectors and appenders.
> >
> >>> Currently they use a decanter specific API but they could easily be
> >>> converted into the more general rcomp api.
> >>> We could also create a bridge to camel components to leverage the
> >>> many existing camel components using the rcomp API as well as
> >>> offering rcomp components to camel.
> >>>
> >>> Components alone are of course not enough. One big strength of
> >Apache
> >>> Camel is the DSL. In case of rcomp I propose to not create our own
> >>> DSL and instead use existing DSLs that work well in OSGi. Two
> >examples:
> >>>
> >>> Akka and reactive streams
> >>>
> >https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
> <https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams>
> >>>
> >>> Reactor and reactive streams
> >>>
> >https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring <https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring>
> >
> >>>
> >>>
> >>> Another integration is with REST. It is already possible to
> >integrate
> >>> CXF Rest services with reactive streams using some adapters but we
> >>> could have native integration.
> >>>
> >>>
> >>> Risks and Opportunities
> >>>
> >>> The main risk I see is not gathering a critical mass of components
> >to
> >>> draw more people.
> >>> Another risk is that the RComponent API or the reactor streams have
> >>> some unexpected limitations.
> >>> The big opportunity I see is that the rcomp API is very simple so
> >the
> >>> barrier of entry is low.
> >>> I also hope that this might become a new foundation for a simpler
> >and
> >>> more modern Apache Camel.
> >>>
> >>> So this all depends on getting some support by you all.
> >>>
> >>> Christian
> >>>
> >>>
> >>
> >
> >--
> >Christian Schneider
> >http://www.liquid-reality.de
> >
> >Open Source Architect
> >http://www.talend.com
>
>
>
>
> --
> --
> Christian Schneider
> http://www.liquid-reality.de
> <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
>
> Computer Scientist
> http://www.adobe.com
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Christian Schneider <ch...@die-schneider.net>.
I have now organized the repo and did the initial push with my current code
of reactive components.
You can find the repo at:
https://github.com/apache/karaf-reactive-components
I would still be happy about a review. I think we should have no licensing
issues as the code was only developed by me and I have an ICLA.
So I can confirm that this is fully apache licensed and written by me.
Please let me know if we need something more formal.
Best
Christian
2017-08-08 14:26 GMT+02:00 Jean-Baptiste Onofré <jb...@nanthrax.net>:
> Hi
>
> I will do a new review round tomorrow morning.
>
> Thanks
> Regards
> JB
>
> On Aug 8, 2017, 13:04, at 13:04, Christian Schneider <
> chris@die-schneider.net> wrote:
> >I now adapted the package names as well as the maven coordinates. All
> >files should now also have the apache headers.
> >I also made sure the build now works without any running mqtt or kafka
> >server.
> >
> >I would be happy about a quick review of the current status. If there
> >are no objections then I will go ahead and ask for a git repository.
> >
> >Christian
> >
> >On 03.08.2017 08:02, Jean-Baptiste Onofré wrote:
> >> Hi Christian,
> >>
> >> the proposal is interesting, and I see a first potential use in
> >> Decanter for sure.
> >>
> >> The comment about Camel is fair as it looks very similar at first
> >glance.
> >>
> >> However, the scope is slightly different IMHO.
> >>
> >> I checked about the legal. The code should be cleanup for donation
> >> (ASF headers, package names, ...). About the dependency license, for
> >> the reactive-streams, it' OK as it uses Creative Commons Zero into
> >the
> >> Public Domain. And Reactor is already under Apache license.
> >>
> >> The DSL is a hot topic. I understand that leveraging Akka or Reactor
> >> is easier, but I'm a bit concern that we could be too much "coupled".
> >> Maybe our own simple DSL could help. Thoughts ?
> >>
> >> Thanks anyway !
> >> Regards
> >> JB
> >>
> >> On 07/19/2017 01:02 PM, Christian Schneider wrote:
> >>>
> >>> Scope
> >>>
> >>> I recently experimented with reactive streams and built a small
> >>> component framework on top of this spec.
> >>> See https://github.com/cschneider/reactive-components
> >>>
> >>> The goal is to have a small API that can encapsulate a protocol and
> >>> transport. The code using such a reactive component should not
> >>> directly depend on the specifics of the transport or protocol.
> >>> Another goal is to have reactive features like back pressure.
> >>> Ultimately I am searching for something like Apache Camel Components
> >
> >>> but with a lot less coupling. In camel the big problem is that
> >>> components depend on camel core which unfortunately is much more
> >than
> >>> a component API. So any camel component is coupled quite tightly to
> >>> all of camel core.
> >>>
> >>>
> >>> Proposal
> >>>
> >>> I propose to donate my code to Apache and establish this as a Apache
> >
> >>> Karaf sub project. Some people like Jean-Baptiste and Hadrian have
> >>> already expressed that they support this and I hope for some more
> >>> feedback and help.
> >>>
> >>> I chose the Karaf project at the moment but am also open to placing
> >>> this in another Apache project. Some matching projects would be
> >>> Apache Camel, Aries or Felix.
> >>>
> >>>
> >>> Component API
> >>>
> >>> I was trying to find the simplest API that would allow similar
> >>> components to camel in one way mode.
> >>>
> >>> public interface RComponent {
> >>> <T> Publisher<T> from(String destination, Class<T> type);
> >>> <T> Subscriber<T> to(String destination, Class<T> type);
> >>> }
> >>>
> >>> A component is a factory for Publishers and Subscribers. From and to
> >
> >>> have similar meaning as in camel. The component can be given a
> >source
> >>> / target type to produce / consume. So with the OSGi Converter spec
> >>> this would allow to have type safe messaging without coding the
> >>> conversion in every component. Each component is exposed as a
> >service
> >>> which encapsulates most of the configuration. All endpoint specific
> >>> configuration can be done using the destination String.
> >>>
> >>> Publisher and Subscriber are interfaces from the reactive streams
> >api
> >>> (http://www.reactive-streams.org/). So they are well defined and
> >have
> >>> zero additional dependencies.
> >>>
> >>> I also considered to use OSGi push streams which is an OSGi spec and
> >
> >>> would also be an interesting foundation. I decided against that
> >>> though as push streams have no API that is separate from the DSL and
> >
> >>> will probably not be used a lot outside of OSGi.
> >>>
> >>> See the examples for how to use this in practice.
> >>> https://github.com/cschneider/reactive-components
> >>>
> >>>
> >>> Possible use cases
> >>>
> >>> Two big use cases are reactive microservices that need messaging as
> >>> well as plain camel like integrations.
> >>> Another case are the Apache Karaf decanter collectors and appenders.
> >
> >>> Currently they use a decanter specific API but they could easily be
> >>> converted into the more general rcomp api.
> >>> We could also create a bridge to camel components to leverage the
> >>> many existing camel components using the rcomp API as well as
> >>> offering rcomp components to camel.
> >>>
> >>> Components alone are of course not enough. One big strength of
> >Apache
> >>> Camel is the DSL. In case of rcomp I propose to not create our own
> >>> DSL and instead use existing DSLs that work well in OSGi. Two
> >examples:
> >>>
> >>> Akka and reactive streams
> >>>
> >https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
> >>>
> >>> Reactor and reactive streams
> >>>
> >https://de.slideshare.net/StphaneMaldini/reactor-30-a-
> reactive-foundation-for-java-8-and-spring
> >
> >>>
> >>>
> >>> Another integration is with REST. It is already possible to
> >integrate
> >>> CXF Rest services with reactive streams using some adapters but we
> >>> could have native integration.
> >>>
> >>>
> >>> Risks and Opportunities
> >>>
> >>> The main risk I see is not gathering a critical mass of components
> >to
> >>> draw more people.
> >>> Another risk is that the RComponent API or the reactor streams have
> >>> some unexpected limitations.
> >>> The big opportunity I see is that the rcomp API is very simple so
> >the
> >>> barrier of entry is low.
> >>> I also hope that this might become a new foundation for a simpler
> >and
> >>> more modern Apache Camel.
> >>>
> >>> So this all depends on getting some support by you all.
> >>>
> >>> Christian
> >>>
> >>>
> >>
> >
> >--
> >Christian Schneider
> >http://www.liquid-reality.de
> >
> >Open Source Architect
> >http://www.talend.com
>
--
--
Christian Schneider
http://www.liquid-reality.de
<https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
Computer Scientist
http://www.adobe.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi
I will do a new review round tomorrow morning.
Thanks
Regards
JB
On Aug 8, 2017, 13:04, at 13:04, Christian Schneider <ch...@die-schneider.net> wrote:
>I now adapted the package names as well as the maven coordinates. All
>files should now also have the apache headers.
>I also made sure the build now works without any running mqtt or kafka
>server.
>
>I would be happy about a quick review of the current status. If there
>are no objections then I will go ahead and ask for a git repository.
>
>Christian
>
>On 03.08.2017 08:02, Jean-Baptiste Onofré wrote:
>> Hi Christian,
>>
>> the proposal is interesting, and I see a first potential use in
>> Decanter for sure.
>>
>> The comment about Camel is fair as it looks very similar at first
>glance.
>>
>> However, the scope is slightly different IMHO.
>>
>> I checked about the legal. The code should be cleanup for donation
>> (ASF headers, package names, ...). About the dependency license, for
>> the reactive-streams, it' OK as it uses Creative Commons Zero into
>the
>> Public Domain. And Reactor is already under Apache license.
>>
>> The DSL is a hot topic. I understand that leveraging Akka or Reactor
>> is easier, but I'm a bit concern that we could be too much "coupled".
>> Maybe our own simple DSL could help. Thoughts ?
>>
>> Thanks anyway !
>> Regards
>> JB
>>
>> On 07/19/2017 01:02 PM, Christian Schneider wrote:
>>>
>>> Scope
>>>
>>> I recently experimented with reactive streams and built a small
>>> component framework on top of this spec.
>>> See https://github.com/cschneider/reactive-components
>>>
>>> The goal is to have a small API that can encapsulate a protocol and
>>> transport. The code using such a reactive component should not
>>> directly depend on the specifics of the transport or protocol.
>>> Another goal is to have reactive features like back pressure.
>>> Ultimately I am searching for something like Apache Camel Components
>
>>> but with a lot less coupling. In camel the big problem is that
>>> components depend on camel core which unfortunately is much more
>than
>>> a component API. So any camel component is coupled quite tightly to
>>> all of camel core.
>>>
>>>
>>> Proposal
>>>
>>> I propose to donate my code to Apache and establish this as a Apache
>
>>> Karaf sub project. Some people like Jean-Baptiste and Hadrian have
>>> already expressed that they support this and I hope for some more
>>> feedback and help.
>>>
>>> I chose the Karaf project at the moment but am also open to placing
>>> this in another Apache project. Some matching projects would be
>>> Apache Camel, Aries or Felix.
>>>
>>>
>>> Component API
>>>
>>> I was trying to find the simplest API that would allow similar
>>> components to camel in one way mode.
>>>
>>> public interface RComponent {
>>> <T> Publisher<T> from(String destination, Class<T> type);
>>> <T> Subscriber<T> to(String destination, Class<T> type);
>>> }
>>>
>>> A component is a factory for Publishers and Subscribers. From and to
>
>>> have similar meaning as in camel. The component can be given a
>source
>>> / target type to produce / consume. So with the OSGi Converter spec
>>> this would allow to have type safe messaging without coding the
>>> conversion in every component. Each component is exposed as a
>service
>>> which encapsulates most of the configuration. All endpoint specific
>>> configuration can be done using the destination String.
>>>
>>> Publisher and Subscriber are interfaces from the reactive streams
>api
>>> (http://www.reactive-streams.org/). So they are well defined and
>have
>>> zero additional dependencies.
>>>
>>> I also considered to use OSGi push streams which is an OSGi spec and
>
>>> would also be an interesting foundation. I decided against that
>>> though as push streams have no API that is separate from the DSL and
>
>>> will probably not be used a lot outside of OSGi.
>>>
>>> See the examples for how to use this in practice.
>>> https://github.com/cschneider/reactive-components
>>>
>>>
>>> Possible use cases
>>>
>>> Two big use cases are reactive microservices that need messaging as
>>> well as plain camel like integrations.
>>> Another case are the Apache Karaf decanter collectors and appenders.
>
>>> Currently they use a decanter specific API but they could easily be
>>> converted into the more general rcomp api.
>>> We could also create a bridge to camel components to leverage the
>>> many existing camel components using the rcomp API as well as
>>> offering rcomp components to camel.
>>>
>>> Components alone are of course not enough. One big strength of
>Apache
>>> Camel is the DSL. In case of rcomp I propose to not create our own
>>> DSL and instead use existing DSLs that work well in OSGi. Two
>examples:
>>>
>>> Akka and reactive streams
>>>
>https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
>>>
>>> Reactor and reactive streams
>>>
>https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring
>
>>>
>>>
>>> Another integration is with REST. It is already possible to
>integrate
>>> CXF Rest services with reactive streams using some adapters but we
>>> could have native integration.
>>>
>>>
>>> Risks and Opportunities
>>>
>>> The main risk I see is not gathering a critical mass of components
>to
>>> draw more people.
>>> Another risk is that the RComponent API or the reactor streams have
>>> some unexpected limitations.
>>> The big opportunity I see is that the rcomp API is very simple so
>the
>>> barrier of entry is low.
>>> I also hope that this might become a new foundation for a simpler
>and
>>> more modern Apache Camel.
>>>
>>> So this all depends on getting some support by you all.
>>>
>>> Christian
>>>
>>>
>>
>
>--
>Christian Schneider
>http://www.liquid-reality.de
>
>Open Source Architect
>http://www.talend.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Christian Schneider <ch...@die-schneider.net>.
I now adapted the package names as well as the maven coordinates. All
files should now also have the apache headers.
I also made sure the build now works without any running mqtt or kafka
server.
I would be happy about a quick review of the current status. If there
are no objections then I will go ahead and ask for a git repository.
Christian
On 03.08.2017 08:02, Jean-Baptiste Onofré wrote:
> Hi Christian,
>
> the proposal is interesting, and I see a first potential use in
> Decanter for sure.
>
> The comment about Camel is fair as it looks very similar at first glance.
>
> However, the scope is slightly different IMHO.
>
> I checked about the legal. The code should be cleanup for donation
> (ASF headers, package names, ...). About the dependency license, for
> the reactive-streams, it' OK as it uses Creative Commons Zero into the
> Public Domain. And Reactor is already under Apache license.
>
> The DSL is a hot topic. I understand that leveraging Akka or Reactor
> is easier, but I'm a bit concern that we could be too much "coupled".
> Maybe our own simple DSL could help. Thoughts ?
>
> Thanks anyway !
> Regards
> JB
>
> On 07/19/2017 01:02 PM, Christian Schneider wrote:
>>
>> Scope
>>
>> I recently experimented with reactive streams and built a small
>> component framework on top of this spec.
>> See https://github.com/cschneider/reactive-components
>>
>> The goal is to have a small API that can encapsulate a protocol and
>> transport. The code using such a reactive component should not
>> directly depend on the specifics of the transport or protocol.
>> Another goal is to have reactive features like back pressure.
>> Ultimately I am searching for something like Apache Camel Components
>> but with a lot less coupling. In camel the big problem is that
>> components depend on camel core which unfortunately is much more than
>> a component API. So any camel component is coupled quite tightly to
>> all of camel core.
>>
>>
>> Proposal
>>
>> I propose to donate my code to Apache and establish this as a Apache
>> Karaf sub project. Some people like Jean-Baptiste and Hadrian have
>> already expressed that they support this and I hope for some more
>> feedback and help.
>>
>> I chose the Karaf project at the moment but am also open to placing
>> this in another Apache project. Some matching projects would be
>> Apache Camel, Aries or Felix.
>>
>>
>> Component API
>>
>> I was trying to find the simplest API that would allow similar
>> components to camel in one way mode.
>>
>> public interface RComponent {
>> <T> Publisher<T> from(String destination, Class<T> type);
>> <T> Subscriber<T> to(String destination, Class<T> type);
>> }
>>
>> A component is a factory for Publishers and Subscribers. From and to
>> have similar meaning as in camel. The component can be given a source
>> / target type to produce / consume. So with the OSGi Converter spec
>> this would allow to have type safe messaging without coding the
>> conversion in every component. Each component is exposed as a service
>> which encapsulates most of the configuration. All endpoint specific
>> configuration can be done using the destination String.
>>
>> Publisher and Subscriber are interfaces from the reactive streams api
>> (http://www.reactive-streams.org/). So they are well defined and have
>> zero additional dependencies.
>>
>> I also considered to use OSGi push streams which is an OSGi spec and
>> would also be an interesting foundation. I decided against that
>> though as push streams have no API that is separate from the DSL and
>> will probably not be used a lot outside of OSGi.
>>
>> See the examples for how to use this in practice.
>> https://github.com/cschneider/reactive-components
>>
>>
>> Possible use cases
>>
>> Two big use cases are reactive microservices that need messaging as
>> well as plain camel like integrations.
>> Another case are the Apache Karaf decanter collectors and appenders.
>> Currently they use a decanter specific API but they could easily be
>> converted into the more general rcomp api.
>> We could also create a bridge to camel components to leverage the
>> many existing camel components using the rcomp API as well as
>> offering rcomp components to camel.
>>
>> Components alone are of course not enough. One big strength of Apache
>> Camel is the DSL. In case of rcomp I propose to not create our own
>> DSL and instead use existing DSLs that work well in OSGi. Two examples:
>>
>> Akka and reactive streams
>> https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
>>
>> Reactor and reactive streams
>> https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring
>>
>>
>> Another integration is with REST. It is already possible to integrate
>> CXF Rest services with reactive streams using some adapters but we
>> could have native integration.
>>
>>
>> Risks and Opportunities
>>
>> The main risk I see is not gathering a critical mass of components to
>> draw more people.
>> Another risk is that the RComponent API or the reactor streams have
>> some unexpected limitations.
>> The big opportunity I see is that the rcomp API is very simple so the
>> barrier of entry is low.
>> I also hope that this might become a new foundation for a simpler and
>> more modern Apache Camel.
>>
>> So this all depends on getting some support by you all.
>>
>> Christian
>>
>>
>
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Luca Burgazzoli <lb...@gmail.com>.
In camel there is now a camel-reactive-streams component that bridges
camel and reactive-streams specs so the integration with the proposed
subproject should be easy.
There is also a camel-reactor and - I hope soon - a camel-rxjava2
component to provide an implementation of the camel-reactive-streams
api based on reactor or rxjava2.
Maybe we could have a new "camel-rcomp" component to ease the integration.
---
Luca Burgazzoli
On Thu, Aug 3, 2017 at 8:02 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi Christian,
>
> the proposal is interesting, and I see a first potential use in Decanter for
> sure.
>
> The comment about Camel is fair as it looks very similar at first glance.
>
> However, the scope is slightly different IMHO.
>
> I checked about the legal. The code should be cleanup for donation (ASF
> headers, package names, ...). About the dependency license, for the
> reactive-streams, it' OK as it uses Creative Commons Zero into the Public
> Domain. And Reactor is already under Apache license.
>
> The DSL is a hot topic. I understand that leveraging Akka or Reactor is
> easier, but I'm a bit concern that we could be too much "coupled".
> Maybe our own simple DSL could help. Thoughts ?
>
> Thanks anyway !
> Regards
> JB
>
>
> On 07/19/2017 01:02 PM, Christian Schneider wrote:
>>
>>
>> Scope
>>
>> I recently experimented with reactive streams and built a small component
>> framework on top of this spec.
>> See https://github.com/cschneider/reactive-components
>>
>> The goal is to have a small API that can encapsulate a protocol and
>> transport. The code using such a reactive component should not directly
>> depend on the specifics of the transport or protocol. Another goal is to
>> have reactive features like back pressure. Ultimately I am searching for
>> something like Apache Camel Components but with a lot less coupling. In
>> camel the big problem is that components depend on camel core which
>> unfortunately is much more than a component API. So any camel component is
>> coupled quite tightly to all of camel core.
>>
>>
>> Proposal
>>
>> I propose to donate my code to Apache and establish this as a Apache Karaf
>> sub project. Some people like Jean-Baptiste and Hadrian have already
>> expressed that they support this and I hope for some more feedback and help.
>>
>> I chose the Karaf project at the moment but am also open to placing this
>> in another Apache project. Some matching projects would be Apache Camel,
>> Aries or Felix.
>>
>>
>> Component API
>>
>> I was trying to find the simplest API that would allow similar components
>> to camel in one way mode.
>>
>> public interface RComponent {
>> <T> Publisher<T> from(String destination, Class<T> type);
>> <T> Subscriber<T> to(String destination, Class<T> type);
>> }
>>
>> A component is a factory for Publishers and Subscribers. From and to have
>> similar meaning as in camel. The component can be given a source / target
>> type to produce / consume. So with the OSGi Converter spec this would allow
>> to have type safe messaging without coding the conversion in every
>> component. Each component is exposed as a service which encapsulates most of
>> the configuration. All endpoint specific configuration can be done using the
>> destination String.
>>
>> Publisher and Subscriber are interfaces from the reactive streams api
>> (http://www.reactive-streams.org/). So they are well defined and have zero
>> additional dependencies.
>>
>> I also considered to use OSGi push streams which is an OSGi spec and would
>> also be an interesting foundation. I decided against that though as push
>> streams have no API that is separate from the DSL and will probably not be
>> used a lot outside of OSGi.
>>
>> See the examples for how to use this in practice.
>> https://github.com/cschneider/reactive-components
>>
>>
>> Possible use cases
>>
>> Two big use cases are reactive microservices that need messaging as well
>> as plain camel like integrations.
>> Another case are the Apache Karaf decanter collectors and appenders.
>> Currently they use a decanter specific API but they could easily be
>> converted into the more general rcomp api.
>> We could also create a bridge to camel components to leverage the many
>> existing camel components using the rcomp API as well as offering rcomp
>> components to camel.
>>
>> Components alone are of course not enough. One big strength of Apache
>> Camel is the DSL. In case of rcomp I propose to not create our own DSL and
>> instead use existing DSLs that work well in OSGi. Two examples:
>>
>> Akka and reactive streams
>> https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
>>
>> Reactor and reactive streams
>>
>> https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring
>>
>> Another integration is with REST. It is already possible to integrate CXF
>> Rest services with reactive streams using some adapters but we could have
>> native integration.
>>
>>
>> Risks and Opportunities
>>
>> The main risk I see is not gathering a critical mass of components to draw
>> more people.
>> Another risk is that the RComponent API or the reactor streams have some
>> unexpected limitations.
>> The big opportunity I see is that the rcomp API is very simple so the
>> barrier of entry is low.
>> I also hope that this might become a new foundation for a simpler and more
>> modern Apache Camel.
>>
>> So this all depends on getting some support by you all.
>>
>> Christian
>>
>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: [Proposal] New subproject rcomp - Reactive components framework
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Christian,
the proposal is interesting, and I see a first potential use in Decanter for sure.
The comment about Camel is fair as it looks very similar at first glance.
However, the scope is slightly different IMHO.
I checked about the legal. The code should be cleanup for donation (ASF headers,
package names, ...). About the dependency license, for the reactive-streams, it'
OK as it uses Creative Commons Zero into the Public Domain. And Reactor is
already under Apache license.
The DSL is a hot topic. I understand that leveraging Akka or Reactor is easier,
but I'm a bit concern that we could be too much "coupled".
Maybe our own simple DSL could help. Thoughts ?
Thanks anyway !
Regards
JB
On 07/19/2017 01:02 PM, Christian Schneider wrote:
>
> Scope
>
> I recently experimented with reactive streams and built a small component
> framework on top of this spec.
> See https://github.com/cschneider/reactive-components
>
> The goal is to have a small API that can encapsulate a protocol and transport.
> The code using such a reactive component should not directly depend on the
> specifics of the transport or protocol. Another goal is to have reactive
> features like back pressure. Ultimately I am searching for something like Apache
> Camel Components but with a lot less coupling. In camel the big problem is that
> components depend on camel core which unfortunately is much more than a
> component API. So any camel component is coupled quite tightly to all of camel
> core.
>
>
> Proposal
>
> I propose to donate my code to Apache and establish this as a Apache Karaf sub
> project. Some people like Jean-Baptiste and Hadrian have already expressed that
> they support this and I hope for some more feedback and help.
>
> I chose the Karaf project at the moment but am also open to placing this in
> another Apache project. Some matching projects would be Apache Camel, Aries or
> Felix.
>
>
> Component API
>
> I was trying to find the simplest API that would allow similar components to
> camel in one way mode.
>
> public interface RComponent {
> <T> Publisher<T> from(String destination, Class<T> type);
> <T> Subscriber<T> to(String destination, Class<T> type);
> }
>
> A component is a factory for Publishers and Subscribers. From and to have
> similar meaning as in camel. The component can be given a source / target type
> to produce / consume. So with the OSGi Converter spec this would allow to have
> type safe messaging without coding the conversion in every component. Each
> component is exposed as a service which encapsulates most of the configuration.
> All endpoint specific configuration can be done using the destination String.
>
> Publisher and Subscriber are interfaces from the reactive streams api
> (http://www.reactive-streams.org/). So they are well defined and have zero
> additional dependencies.
>
> I also considered to use OSGi push streams which is an OSGi spec and would also
> be an interesting foundation. I decided against that though as push streams have
> no API that is separate from the DSL and will probably not be used a lot outside
> of OSGi.
>
> See the examples for how to use this in practice.
> https://github.com/cschneider/reactive-components
>
>
> Possible use cases
>
> Two big use cases are reactive microservices that need messaging as well as
> plain camel like integrations.
> Another case are the Apache Karaf decanter collectors and appenders. Currently
> they use a decanter specific API but they could easily be converted into the
> more general rcomp api.
> We could also create a bridge to camel components to leverage the many existing
> camel components using the rcomp API as well as offering rcomp components to camel.
>
> Components alone are of course not enough. One big strength of Apache Camel is
> the DSL. In case of rcomp I propose to not create our own DSL and instead use
> existing DSLs that work well in OSGi. Two examples:
>
> Akka and reactive streams
> https://de.slideshare.net/ktoso/reactive-integrations-with-akka-streams
>
> Reactor and reactive streams
> https://de.slideshare.net/StphaneMaldini/reactor-30-a-reactive-foundation-for-java-8-and-spring
>
>
> Another integration is with REST. It is already possible to integrate CXF Rest
> services with reactive streams using some adapters but we could have native
> integration.
>
>
> Risks and Opportunities
>
> The main risk I see is not gathering a critical mass of components to draw more
> people.
> Another risk is that the RComponent API or the reactor streams have some
> unexpected limitations.
> The big opportunity I see is that the rcomp API is very simple so the barrier of
> entry is low.
> I also hope that this might become a new foundation for a simpler and more
> modern Apache Camel.
>
> So this all depends on getting some support by you all.
>
> Christian
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com