You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Darren Govoni <da...@ontrenet.com> on 2020/04/24 11:05:27 UTC
Can Nifi load balance flowfiles?
Hi
Let's say I have a splitjson processor. I want to connect 10 processors to it such that it will send one output to one processor in an evenly distributed manner.
Can Nifi do this?
Darren
Sent from my Verizon, Samsung Galaxy smartphone
Re: Can Nifi load balance flowfiles?
Posted by Darren Govoni <da...@ontrenet.com>.
Thanks Joe.
I'm shooting for [CASE1].
The part I missed was setting the number of relationships. I just dragged connections from it thinking the strategy applied.
My regrets and thank you for the clarification!
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Friday, April 24, 2020 2:31 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Can Nifi load balance flowfiles?
Darren,
It isn't quite clear to me what you want to do. Is the pattern
FlowFile A and B enter DistributeLoad which has relationships 1 and 2.
[CASE1] A goes to 1, B goes to 2.
OR
[CASE2] is is A goes to 1, B goes to 1 and CopyA goes to 2, CopyB goes to 2?
If it is CASE1 you want distribute load and you set the 'number of relationships' property to 2 and the strategy you want. Then you set relationship 1 to some target processor and relationship 2 to some other processor.
If it is CASE2 you want then you just use whatever source processor you have before DistributeLoad and you use the same relationship (for example success) more than once and we'll make flowfile copies for you.
Hopefully that helps.
Thanks
On Fri, Apr 24, 2020 at 2:19 PM Darren Govoni <da...@ontrenet.com>> wrote:
Hi Joe,
I've tried to use DistributeLoad, but perhaps I am doing something wrong.
I have 2 flowfiles coming into DistributeLoad and I want that processor to distribute them evenly across its outbound connections.
I have 2 connections outbound from DistributeLoad. I expected to see 1 flowfile in each outbound queue, but instead I see 2 in each, like a normal processor would distribute flowfiles.
Is there a special use of this processor to have to evenly distribute the received flowfiles across outbound queues?
thanks!!
________________________________
From: Joe Witt <jo...@gmail.com>>
Sent: Friday, April 24, 2020 8:08 AM
To: users@nifi.apache.org<ma...@nifi.apache.org> <us...@nifi.apache.org>>
Subject: Re: Can Nifi load balance flowfiles?
Take a look at DistributeLoad.
thanks
On Fri, Apr 24, 2020 at 7:05 AM Darren Govoni <da...@ontrenet.com>> wrote:
Hi
Let's say I have a splitjson processor. I want to connect 10 processors to it such that it will send one output to one processor in an evenly distributed manner.
Can Nifi do this?
Darren
Sent from my Verizon, Samsung Galaxy smartphone
Re: Can Nifi load balance flowfiles?
Posted by Joe Witt <jo...@gmail.com>.
Darren,
It isn't quite clear to me what you want to do. Is the pattern
FlowFile A and B enter DistributeLoad which has relationships 1 and 2.
[CASE1] A goes to 1, B goes to 2.
OR
[CASE2] is is A goes to 1, B goes to 1 and CopyA goes to 2, CopyB goes to 2?
If it is CASE1 you want distribute load and you set the 'number of
relationships' property to 2 and the strategy you want. Then you set
relationship 1 to some target processor and relationship 2 to some other
processor.
If it is CASE2 you want then you just use whatever source processor you
have before DistributeLoad and you use the same relationship (for example
success) more than once and we'll make flowfile copies for you.
Hopefully that helps.
Thanks
On Fri, Apr 24, 2020 at 2:19 PM Darren Govoni <da...@ontrenet.com> wrote:
> Hi Joe,
> I've tried to use DistributeLoad, but perhaps I am doing something
> wrong.
>
> I have 2 flowfiles coming into DistributeLoad and I want that processor to
> distribute them evenly across its outbound connections.
>
> I have 2 connections outbound from DistributeLoad. I expected to see 1
> flowfile in each outbound queue, but instead I see 2 in each, like a normal
> processor would distribute flowfiles.
>
> Is there a special use of this processor to have to evenly distribute the
> received flowfiles across outbound queues?
>
> thanks!!
> ------------------------------
> *From:* Joe Witt <jo...@gmail.com>
> *Sent:* Friday, April 24, 2020 8:08 AM
> *To:* users@nifi.apache.org <us...@nifi.apache.org>
> *Subject:* Re: Can Nifi load balance flowfiles?
>
> Take a look at DistributeLoad.
>
> thanks
>
> On Fri, Apr 24, 2020 at 7:05 AM Darren Govoni <da...@ontrenet.com> wrote:
>
> Hi
>
> Let's say I have a splitjson processor. I want to connect 10 processors to
> it such that it will send one output to one processor in an evenly
> distributed manner.
>
> Can Nifi do this?
>
> Darren
>
> Sent from my Verizon, Samsung Galaxy smartphone
>
>
Re: Can Nifi load balance flowfiles?
Posted by Darren Govoni <da...@ontrenet.com>.
Hi Joe,
I've tried to use DistributeLoad, but perhaps I am doing something wrong.
I have 2 flowfiles coming into DistributeLoad and I want that processor to distribute them evenly across its outbound connections.
I have 2 connections outbound from DistributeLoad. I expected to see 1 flowfile in each outbound queue, but instead I see 2 in each, like a normal processor would distribute flowfiles.
Is there a special use of this processor to have to evenly distribute the received flowfiles across outbound queues?
thanks!!
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Friday, April 24, 2020 8:08 AM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Can Nifi load balance flowfiles?
Take a look at DistributeLoad.
thanks
On Fri, Apr 24, 2020 at 7:05 AM Darren Govoni <da...@ontrenet.com>> wrote:
Hi
Let's say I have a splitjson processor. I want to connect 10 processors to it such that it will send one output to one processor in an evenly distributed manner.
Can Nifi do this?
Darren
Sent from my Verizon, Samsung Galaxy smartphone
Re: Can Nifi load balance flowfiles?
Posted by Joe Witt <jo...@gmail.com>.
Take a look at DistributeLoad.
thanks
On Fri, Apr 24, 2020 at 7:05 AM Darren Govoni <da...@ontrenet.com> wrote:
> Hi
>
> Let's say I have a splitjson processor. I want to connect 10 processors to
> it such that it will send one output to one processor in an evenly
> distributed manner.
>
> Can Nifi do this?
>
> Darren
>
> Sent from my Verizon, Samsung Galaxy smartphone
>