site-to-site timeout

[Mark Bean <ma...@gmail.com>]

It was observed in Apache NiFi 0.x that when the NCM goes down or becomes
unavailable, data being sent to the cluster via site-to-site continues to
flow to the Nodes for 24 hours. There is a state file
NIFI_HOME/conf/state/{RPG-UUID}.peers. When this file becomes > 24 hours
old, the site-to-site communication finally stops.

First, can you confirm this is an accurate observation? Second, where in
the code can I find this? Lastly, and most importantly, does this behavior
apply to 1.x since there is no longer an NCM?

Thanks,
Mark

Re: site-to-site timeout

[Mark Bean <ma...@gmail.com>]

Thanks for confirming. As this question was mostly for 1.x, I'm happy to
hear (and further read into your statement) that site-to-site will
continuously function on surviving Nodes even if one or more Nodes go down.

Thanks,
Mark


On Mon, Jan 8, 2018 at 1:58 PM, Mark Payne <ma...@hotmail.com> wrote:

> Mark,
>
> That does sound correct. The code would be probably be either in the
> nifi-site-to-site-client module
> or the in the StandardRemoteGroupPort, within the nifi-framework-core
> module. It's been a while since
> I've looked at 0.x, so it's hard to say for sure.
>
> In 1.x, this is a non-issue. The site-to-site client will iterate through
> all nodes in the remote cluster, attempting
> to connect to each of them to determine the cluster topology. If it fails
> to connect to any of them, it will then
> try again after a bit, until it is finally successful.
>
> Thanks
> -Mark
>
>
> > On Jan 8, 2018, at 1:29 PM, Mark Bean <ma...@gmail.com> wrote:
> >
> > It was observed in Apache NiFi 0.x that when the NCM goes down or becomes
> > unavailable, data being sent to the cluster via site-to-site continues to
> > flow to the Nodes for 24 hours. There is a state file
> > NIFI_HOME/conf/state/{RPG-UUID}.peers. When this file becomes > 24 hours
> > old, the site-to-site communication finally stops.
> >
> > First, can you confirm this is an accurate observation? Second, where in
> > the code can I find this? Lastly, and most importantly, does this
> behavior
> > apply to 1.x since there is no longer an NCM?
> >
> > Thanks,
> > Mark
>
>

Re: site-to-site timeout

[Mark Payne <ma...@hotmail.com>]

Mark,

That does sound correct. The code would be probably be either in the nifi-site-to-site-client module
or the in the StandardRemoteGroupPort, within the nifi-framework-core module. It's been a while since
I've looked at 0.x, so it's hard to say for sure.

In 1.x, this is a non-issue. The site-to-site client will iterate through all nodes in the remote cluster, attempting
to connect to each of them to determine the cluster topology. If it fails to connect to any of them, it will then
try again after a bit, until it is finally successful.

Thanks
-Mark


> On Jan 8, 2018, at 1:29 PM, Mark Bean <ma...@gmail.com> wrote:
> 
> It was observed in Apache NiFi 0.x that when the NCM goes down or becomes
> unavailable, data being sent to the cluster via site-to-site continues to
> flow to the Nodes for 24 hours. There is a state file
> NIFI_HOME/conf/state/{RPG-UUID}.peers. When this file becomes > 24 hours
> old, the site-to-site communication finally stops.
> 
> First, can you confirm this is an accurate observation? Second, where in
> the code can I find this? Lastly, and most importantly, does this behavior
> apply to 1.x since there is no longer an NCM?
> 
> Thanks,
> Mark