Maximum pipeline size to be handled by user

[Patrick Wiener <wi...@apache.org>]

Hi all,

Aside from the technical topics we discuss here, I’m always quite curious about experiences the from a user’s perspective when using StreamPipes.
As this one also hits me quite a lot when modeling pipelines myself, I am eager to see what your view is on this one:

What do you think is the „maximum“ pipeline size in terms of pipeline elements to be handled/modeled by a user (if any) without getting too complex? 

[1] less than 5 pipeline element
[2] between 5-10 pipeline element
[3] more than 10 pipeline element

Again, this is not a technical question and it depends on the use case and entailed logic within an individual pipeline element. But the greater a pipeline
gets, the more complex it gets, making it hard to quickly assess what’s actually going on.

Don’t be too philosophical on this one :) just share your experience or an estimate what your common pipeline sizes are.

Cheers
Patrick

Re: Maximum pipeline size to be handled by user

[Philipp Zehnder <ze...@apache.org>]

Hi Patrick,

great question. For me I would say it is between 5-10 pipeline elements.
But it also highly depends on the use case. When I have a production line with different stations I prefer to have a single pipeline per station, even if that means there are many pipeline elements.
I think it is easier to manage when I directly know which pipeline I must edit, instead of having multiple pipelines with one function each.

Philipp

> On 28. Sep 2021, at 10:00, Patrick Wiener <wi...@apache.org> wrote:
> 
> Hi all,
> 
> Aside from the technical topics we discuss here, I’m always quite curious about experiences the from a user’s perspective when using StreamPipes.
> As this one also hits me quite a lot when modeling pipelines myself, I am eager to see what your view is on this one:
> 
> What do you think is the „maximum“ pipeline size in terms of pipeline elements to be handled/modeled by a user (if any) without getting too complex? 
> 
> [1] less than 5 pipeline element
> [2] between 5-10 pipeline element
> [3] more than 10 pipeline element
> 
> Again, this is not a technical question and it depends on the use case and entailed logic within an individual pipeline element. But the greater a pipeline
> gets, the more complex it gets, making it hard to quickly assess what’s actually going on.
> 
> Don’t be too philosophical on this one :) just share your experience or an estimate what your common pipeline sizes are.
> 
> Cheers
> Patrick
>