You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Anil Rai <an...@gmail.com> on 2017/03/07 19:48:34 UTC
Request flow within a cluster
Team,
Is there any documentation that explains how Nifi processes a request once
it reach nifi in a cluster settings?
Regards
Anil
Re: Request flow within a cluster
Posted by Anil Rai <an...@gmail.com>.
Thank you.
Is there any documentation on the ideal cluster set up for production
environment? I saw the 3 zoo keepers on 3 VM's and 3 nodes on 3 VM's is the
best. Could you confirm?
Also the ideal system requirement for each node and each zoo keeper.....in
terms of CPU, memory, disk space etc?
and finally archiving....logs, flow files etc and their frequency
Thanks
Anil
On Tue, Mar 7, 2017 at 3:11 PM, Bryan Bende <bb...@gmail.com> wrote:
> Anil,
>
> The short answer is that each node in the cluster is running the
> entire data flow you see in UI, and NiFi does not transfer flow files
> between nodes unless the data flow is created to do that.
>
> This article describes approaches to distributing data across the
> cluster, which is dependent on how the data is entering the cluster:
> https://community.hortonworks.com/articles/16120/how-do-i-
> distribute-data-across-a-nifi-cluster.html
>
> Thanks,
>
> Bryan
>
>
> On Tue, Mar 7, 2017 at 3:03 PM, Anil Rai <an...@gmail.com> wrote:
> > Thanks Andy. I am looking for the processing of a flow file.If i create a
> > flow, with multiple processor, how does this request flow in nifi in a
> > cluster.
> >
> >
> > On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <al...@apache.org>
> wrote:
> >
> >> Anil,
> >>
> >> Are you talking about processing a flowfile or responding to a REST HTTP
> >> request? The Admin Guide section on clustering [1] offers more
> information
> >> but if you can clarify your question you may get better answers from the
> >> mailing list.
> >>
> >> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
> >> guide.html#clustering
> >>
> >>
> >> Andy LoPresto
> >> alopresto@apache.org
> >> *alopresto.apache@gmail.com <al...@gmail.com>*
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>
> >> On Mar 7, 2017, at 11:48 AM, Anil Rai <an...@gmail.com> wrote:
> >>
> >> Team,
> >>
> >> Is there any documentation that explains how Nifi processes a request
> once
> >> it reach nifi in a cluster settings?
> >>
> >> Regards
> >> Anil
> >>
> >>
> >>
>
Re: Request flow within a cluster
Posted by Bryan Bende <bb...@gmail.com>.
Anil,
The short answer is that each node in the cluster is running the
entire data flow you see in UI, and NiFi does not transfer flow files
between nodes unless the data flow is created to do that.
This article describes approaches to distributing data across the
cluster, which is dependent on how the data is entering the cluster:
https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html
Thanks,
Bryan
On Tue, Mar 7, 2017 at 3:03 PM, Anil Rai <an...@gmail.com> wrote:
> Thanks Andy. I am looking for the processing of a flow file.If i create a
> flow, with multiple processor, how does this request flow in nifi in a
> cluster.
>
>
> On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <al...@apache.org> wrote:
>
>> Anil,
>>
>> Are you talking about processing a flowfile or responding to a REST HTTP
>> request? The Admin Guide section on clustering [1] offers more information
>> but if you can clarify your question you may get better answers from the
>> mailing list.
>>
>> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
>> guide.html#clustering
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <al...@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>> On Mar 7, 2017, at 11:48 AM, Anil Rai <an...@gmail.com> wrote:
>>
>> Team,
>>
>> Is there any documentation that explains how Nifi processes a request once
>> it reach nifi in a cluster settings?
>>
>> Regards
>> Anil
>>
>>
>>
Re: Request flow within a cluster
Posted by Anil Rai <an...@gmail.com>.
Thanks Andy. I am looking for the processing of a flow file.If i create a
flow, with multiple processor, how does this request flow in nifi in a
cluster.
On Tue, Mar 7, 2017 at 2:58 PM, Andy LoPresto <al...@apache.org> wrote:
> Anil,
>
> Are you talking about processing a flowfile or responding to a REST HTTP
> request? The Admin Guide section on clustering [1] offers more information
> but if you can clarify your question you may get better answers from the
> mailing list.
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
> guide.html#clustering
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Mar 7, 2017, at 11:48 AM, Anil Rai <an...@gmail.com> wrote:
>
> Team,
>
> Is there any documentation that explains how Nifi processes a request once
> it reach nifi in a cluster settings?
>
> Regards
> Anil
>
>
>
Re: Request flow within a cluster
Posted by Andy LoPresto <al...@apache.org>.
Anil,
Are you talking about processing a flowfile or responding to a REST HTTP request? The Admin Guide section on clustering [1] offers more information but if you can clarify your question you may get better answers from the mailing list.
[1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Mar 7, 2017, at 11:48 AM, Anil Rai <an...@gmail.com> wrote:
>
> Team,
>
> Is there any documentation that explains how Nifi processes a request once
> it reach nifi in a cluster settings?
>
> Regards
> Anil
Re: Request flow within a cluster
Posted by Bryan Bende <bb...@gmail.com>.
Hello,
When you say "request" are you referring to the end-user interacting
with the NiFi web UI which makes a request to the server?
Or are you referring to data entering a NiFi cluster from an external source?
Thanks,
Bryan
On Tue, Mar 7, 2017 at 2:48 PM, Anil Rai <an...@gmail.com> wrote:
> Team,
>
> Is there any documentation that explains how Nifi processes a request once
> it reach nifi in a cluster settings?
>
> Regards
> Anil