3.4.6 rolling restart failing with "Have smaller server identifier"

[Irfan Hamid <ih...@salesforce.com>]

Hi,

I have a 5 node ensemble on which I restarted server.1. Ever since it is
unable to rejoin the quorum. I've even tried bringing everything down and
restarting all the servers (starting at server.1 and working my way up to
server.5) but still no joy.

The log messages are of the form:

xxxxx- Have smaller server identifier, so dropping the connection: (2, 1)
xxxxx- Have smaller server identifier, so dropping the connection: (3, 1)
xxxxx- Have smaller server identifier, so dropping the connection: (4, 1)
xxxxx- Have smaller server identifier, so dropping the connection: (5, 1)

I've looked and it seems like 3.4.6 has the fix for this
via ZOOKEEPER-1805. So what am I doing wrong/how can I fix it?

TIA,
Irfan.

Re: 3.4.6 rolling restart failing with "Have smaller server identifier"

[Michael Han <ha...@cloudera.com>]

What if the rolling restart is performed from highest sid to lowest sid -
5, 4, 3, 2, 1 in this case - would this work?

On Wed, Aug 31, 2016 at 12:25 PM, Irfan Hamid <ih...@salesforce.com> wrote:

> Hi,
>
> I have a 5 node ensemble on which I restarted server.1. Ever since it is
> unable to rejoin the quorum. I've even tried bringing everything down and
> restarting all the servers (starting at server.1 and working my way up to
> server.5) but still no joy.
>
> The log messages are of the form:
>
> xxxxx- Have smaller server identifier, so dropping the connection: (2, 1)
> xxxxx- Have smaller server identifier, so dropping the connection: (3, 1)
> xxxxx- Have smaller server identifier, so dropping the connection: (4, 1)
> xxxxx- Have smaller server identifier, so dropping the connection: (5, 1)
>
> I've looked and it seems like 3.4.6 has the fix for this
> via ZOOKEEPER-1805. So what am I doing wrong/how can I fix it?
>
> TIA,
> Irfan.
>



-- 
Cheers
Michael.