You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Joe Gresock <jg...@gmail.com> on 2017/02/11 13:45:15 UTC
Node in cluster cannot reconnect
I've run into an issue in a 7-node cluster on version 1.1.0 where one of my
nodes can no longer reconnect to the cluster. Bringing up the console on
any other node works fine, but this one claims that "Cluster is still in
the process of voting on the appropriate Data Flow". The flow.xml.gz stays
a 0-byte file after restart, and I don't see anything being written to the
logs other than an occasional checkpoint of the write ahead logs.
I've cleared the $NIFI_HOME/state/local directory and $NIFI_HOME/conf/state
directory, and restarted several times to no avail.
How should I go about getting it to realize that the cluster is no longer
voting?
--
I know what it is to be in need, and I know what it is to have plenty. I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want. I can do
all this through him who gives me strength. *-Philippians 4:12-13*
Re: Node in cluster cannot reconnect
Posted by Joe Gresock <jg...@gmail.com>.
Whew.. it finally reconnected over the weekend.
On Mon, Feb 13, 2017 at 8:19 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Hi Joe,
>
> I really don't know then... I'll defer to guys having a deeper knowledge of
> the clustering code. But that looks like a bug.
>
> Pierre
>
> 2017-02-11 15:02 GMT+01:00 Joe Gresock <jg...@gmail.com>:
>
> > Yeah, that was the first thing I tried, but I just tried again and it
> just
> > recreates the 0byte file and still doesn't connect to the cluster. It
> does
> > appear to load the flow in memory, because I see a warning about a
> > particular "ghost" processor (which is expected at the moment), and I
> don't
> > see how it could have done this without having loaded the flow into
> memory.
> >
> > On Sat, Feb 11, 2017 at 1:54 PM, Pierre Villard <
> > pierre.villard.fr@gmail.com
> > > wrote:
> >
> > > Hi Joe,
> > >
> > > I don't think that would help based on the log message you are seeing,
> > > but... just in case... could you delete the flow.xml.gz file on the
> > > disconnected instance and restart it?
> > >
> > > Pierre
> > >
> > > 2017-02-11 14:45 GMT+01:00 Joe Gresock <jg...@gmail.com>:
> > >
> > > > I've run into an issue in a 7-node cluster on version 1.1.0 where one
> > of
> > > my
> > > > nodes can no longer reconnect to the cluster. Bringing up the
> console
> > on
> > > > any other node works fine, but this one claims that "Cluster is still
> > in
> > > > the process of voting on the appropriate Data Flow". The flow.xml.gz
> > > stays
> > > > a 0-byte file after restart, and I don't see anything being written
> to
> > > the
> > > > logs other than an occasional checkpoint of the write ahead logs.
> > > >
> > > > I've cleared the $NIFI_HOME/state/local directory and
> > > $NIFI_HOME/conf/state
> > > > directory, and restarted several times to no avail.
> > > >
> > > > How should I go about getting it to realize that the cluster is no
> > longer
> > > > voting?
> > > >
> > > > --
> > > > I know what it is to be in need, and I know what it is to have
> > plenty. I
> > > > have learned the secret of being content in any and every situation,
> > > > whether well fed or hungry, whether living in plenty or in want. I
> can
> > > do
> > > > all this through him who gives me strength. *-Philippians 4:12-13*
> > > >
> > >
> >
> >
> >
> > --
> > I know what it is to be in need, and I know what it is to have plenty. I
> > have learned the secret of being content in any and every situation,
> > whether well fed or hungry, whether living in plenty or in want. I can
> do
> > all this through him who gives me strength. *-Philippians 4:12-13*
> >
>
--
I know what it is to be in need, and I know what it is to have plenty. I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want. I can do
all this through him who gives me strength. *-Philippians 4:12-13*
Re: Node in cluster cannot reconnect
Posted by Pierre Villard <pi...@gmail.com>.
Hi Joe,
I really don't know then... I'll defer to guys having a deeper knowledge of
the clustering code. But that looks like a bug.
Pierre
2017-02-11 15:02 GMT+01:00 Joe Gresock <jg...@gmail.com>:
> Yeah, that was the first thing I tried, but I just tried again and it just
> recreates the 0byte file and still doesn't connect to the cluster. It does
> appear to load the flow in memory, because I see a warning about a
> particular "ghost" processor (which is expected at the moment), and I don't
> see how it could have done this without having loaded the flow into memory.
>
> On Sat, Feb 11, 2017 at 1:54 PM, Pierre Villard <
> pierre.villard.fr@gmail.com
> > wrote:
>
> > Hi Joe,
> >
> > I don't think that would help based on the log message you are seeing,
> > but... just in case... could you delete the flow.xml.gz file on the
> > disconnected instance and restart it?
> >
> > Pierre
> >
> > 2017-02-11 14:45 GMT+01:00 Joe Gresock <jg...@gmail.com>:
> >
> > > I've run into an issue in a 7-node cluster on version 1.1.0 where one
> of
> > my
> > > nodes can no longer reconnect to the cluster. Bringing up the console
> on
> > > any other node works fine, but this one claims that "Cluster is still
> in
> > > the process of voting on the appropriate Data Flow". The flow.xml.gz
> > stays
> > > a 0-byte file after restart, and I don't see anything being written to
> > the
> > > logs other than an occasional checkpoint of the write ahead logs.
> > >
> > > I've cleared the $NIFI_HOME/state/local directory and
> > $NIFI_HOME/conf/state
> > > directory, and restarted several times to no avail.
> > >
> > > How should I go about getting it to realize that the cluster is no
> longer
> > > voting?
> > >
> > > --
> > > I know what it is to be in need, and I know what it is to have
> plenty. I
> > > have learned the secret of being content in any and every situation,
> > > whether well fed or hungry, whether living in plenty or in want. I can
> > do
> > > all this through him who gives me strength. *-Philippians 4:12-13*
> > >
> >
>
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty. I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want. I can do
> all this through him who gives me strength. *-Philippians 4:12-13*
>
Re: Node in cluster cannot reconnect
Posted by Joe Gresock <jg...@gmail.com>.
Yeah, that was the first thing I tried, but I just tried again and it just
recreates the 0byte file and still doesn't connect to the cluster. It does
appear to load the flow in memory, because I see a warning about a
particular "ghost" processor (which is expected at the moment), and I don't
see how it could have done this without having loaded the flow into memory.
On Sat, Feb 11, 2017 at 1:54 PM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Hi Joe,
>
> I don't think that would help based on the log message you are seeing,
> but... just in case... could you delete the flow.xml.gz file on the
> disconnected instance and restart it?
>
> Pierre
>
> 2017-02-11 14:45 GMT+01:00 Joe Gresock <jg...@gmail.com>:
>
> > I've run into an issue in a 7-node cluster on version 1.1.0 where one of
> my
> > nodes can no longer reconnect to the cluster. Bringing up the console on
> > any other node works fine, but this one claims that "Cluster is still in
> > the process of voting on the appropriate Data Flow". The flow.xml.gz
> stays
> > a 0-byte file after restart, and I don't see anything being written to
> the
> > logs other than an occasional checkpoint of the write ahead logs.
> >
> > I've cleared the $NIFI_HOME/state/local directory and
> $NIFI_HOME/conf/state
> > directory, and restarted several times to no avail.
> >
> > How should I go about getting it to realize that the cluster is no longer
> > voting?
> >
> > --
> > I know what it is to be in need, and I know what it is to have plenty. I
> > have learned the secret of being content in any and every situation,
> > whether well fed or hungry, whether living in plenty or in want. I can
> do
> > all this through him who gives me strength. *-Philippians 4:12-13*
> >
>
--
I know what it is to be in need, and I know what it is to have plenty. I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want. I can do
all this through him who gives me strength. *-Philippians 4:12-13*
Re: Node in cluster cannot reconnect
Posted by Pierre Villard <pi...@gmail.com>.
Hi Joe,
I don't think that would help based on the log message you are seeing,
but... just in case... could you delete the flow.xml.gz file on the
disconnected instance and restart it?
Pierre
2017-02-11 14:45 GMT+01:00 Joe Gresock <jg...@gmail.com>:
> I've run into an issue in a 7-node cluster on version 1.1.0 where one of my
> nodes can no longer reconnect to the cluster. Bringing up the console on
> any other node works fine, but this one claims that "Cluster is still in
> the process of voting on the appropriate Data Flow". The flow.xml.gz stays
> a 0-byte file after restart, and I don't see anything being written to the
> logs other than an occasional checkpoint of the write ahead logs.
>
> I've cleared the $NIFI_HOME/state/local directory and $NIFI_HOME/conf/state
> directory, and restarted several times to no avail.
>
> How should I go about getting it to realize that the cluster is no longer
> voting?
>
> --
> I know what it is to be in need, and I know what it is to have plenty. I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want. I can do
> all this through him who gives me strength. *-Philippians 4:12-13*
>