Alternative Pub Sub mechanism between Operators

[Gaurav Gupta <ga...@datatorrent.com>]

Currently BufferServer is the only mechanism to transfer data between operators. Does it make sense to make this module pluggable and let users flip it with their implementation if any. Default behavior will be to use current BufferServer.

Thanks
- Gaurav

Re: Alternative Pub Sub mechanism between Operators

[Thomas Weise <th...@datatorrent.com>]

A service provider interface and factory mechanism can be used to swap out
the entire implementation, but more likely to customize some aspects of the
implementation (security etc.).

On Mon, Sep 14, 2015 at 11:35 PM, Yogi Devendra <de...@datatorrent.com>
wrote:

> Can we say that proposed module interface can be implemented by some other
> pub-sub mechanism say Kafka for example?
>
> If yes; then it would be a huge plus to make this modular.
>
> ~ Yogi
>
> On Tue, Sep 15, 2015 at 5:42 AM, Gaurav Gupta <ga...@datatorrent.com>
> wrote:
>
> > Currently BufferServer is the only mechanism to transfer data between
> > operators. Does it make sense to make this module pluggable and let users
> > flip it with their implementation if any. Default behavior will be to use
> > current BufferServer.
> >
> > Thanks
> > - Gaurav
> >
> >
>

Re: Alternative Pub Sub mechanism between Operators

[Yogi Devendra <de...@datatorrent.com>]

Can we say that proposed module interface can be implemented by some other
pub-sub mechanism say Kafka for example?

If yes; then it would be a huge plus to make this modular.

~ Yogi

On Tue, Sep 15, 2015 at 5:42 AM, Gaurav Gupta <ga...@datatorrent.com>
wrote:

> Currently BufferServer is the only mechanism to transfer data between
> operators. Does it make sense to make this module pluggable and let users
> flip it with their implementation if any. Default behavior will be to use
> current BufferServer.
>
> Thanks
> - Gaurav
>
>