Extensions services

[Dominik Riemer <ri...@apache.org>]

Hi all,

 

currently, we have several base services that can be used to develop
adapters + pipeline elements:

streampipes-container-standalone (supports data processors and sinks)

streampipes-container-connect-worker (for adapters)

streampipes-container-extensions (for adapters, data processors and sinks,
but this seems to be more like a quick fix and duplicates code from the
other two modules)

 

Is there any use case where we want an extensions container to not support
adapters, processors and sinks at the same time?

As I'm currently doing the rather large refactoring related to service
discovery where I would need to touch all these modules, I tend towards
having one single service that supports development of all extensions - what
do you think?

 

Dominik

 

 

Re: Extensions services

[Philipp Zehnder <ze...@apache.org>]

Hi Dominik,

yes I like the idea of having a harmonized container for all extensions.
I think the only reason for this is that it has grown historically.

We should discuss how we deploy the system. Do we have a single extensions container with all the adapters, processors, and sinks or do we keep our current approach and have multiple of them.
I am in favor of having multiple running containers with extensions, but then we need a way how to deal with two instances of the same type.
How are we currently dealing with it? I think Patrick has a good solution for this in the edge-extensions branch.

Philipp



> On 18. Jun 2021, at 21:34, Dominik Riemer <ri...@apache.org> wrote:
> 
> Hi all,
> 
> 
> 
> currently, we have several base services that can be used to develop
> adapters + pipeline elements:
> 
> streampipes-container-standalone (supports data processors and sinks)
> 
> streampipes-container-connect-worker (for adapters)
> 
> streampipes-container-extensions (for adapters, data processors and sinks,
> but this seems to be more like a quick fix and duplicates code from the
> other two modules)
> 
> 
> 
> Is there any use case where we want an extensions container to not support
> adapters, processors and sinks at the same time?
> 
> As I'm currently doing the rather large refactoring related to service
> discovery where I would need to touch all these modules, I tend towards
> having one single service that supports development of all extensions - what
> do you think?
> 
> 
> 
> Dominik
> 
> 
> 
> 
>