reusing node ids

[Wes Chow <we...@chartbeat.com>]

How reusable are node ids? Specifically:

1. Node goes down and loses all data in its local Kafka logs. I bring it 
back up, same hostname, same IP, same node id, but with empty logs. Does 
it properly sync with the replicas and continue operation?

2. Node goes down and I replace it with a new machine that has a 
different IP but same hostname and node id. Does it sync and continue?

3. Node goes down and I replace it with a new machine that has a 
different IP and different hostname, but same node id. Does it sync and 
continue?

4. Node stays up but changes hostname. It retains the same IP and same 
node id. Does it keep running?

5. Node goes down and we rebalance partitions, effectively retiring that 
node id. At a much later date, we bring up a new machine. Can we assign 
it that old node id? Does that "later date" have anything to do with 
retention history, or is it irrelevant?

Thanks,
Wes

Re: reusing node ids

[Gwen Shapira <gs...@cloudera.com>]

On Thu, Apr 2, 2015 at 12:19 PM, Wes Chow <we...@chartbeat.com> wrote:

>
> How reusable are node ids? Specifically:
>
> 1. Node goes down and loses all data in its local Kafka logs. I bring it
> back up, same hostname, same IP, same node id, but with empty logs. Does it
> properly sync with the replicas and continue operation?
>

Yes.


>
> 2. Node goes down and I replace it with a new machine that has a different
> IP but same hostname and node id. Does it sync and continue?
>
> 3. Node goes down and I replace it with a new machine that has a different
> IP and different hostname, but same node id. Does it sync and continue?
>


Yes and yes. Its the id that gets registered in zookeeper and counts.


>
> 4. Node stays up but changes hostname. It retains the same IP and same
> node id. Does it keep running?
>

mmmm... good question. I believe it will, but if you used the hostname in
any configuration - other brokers / producers / consumers may have trouble
finding it.


>
> 5. Node goes down and we rebalance partitions, effectively retiring that
> node id. At a much later date, we bring up a new machine. Can we assign it
> that old node id? Does that "later date" have anything to do with retention
> history, or is it irrelevant?
>

If nothing is assigned to that ID any more, it should be safe to reuse. It
has nothing to do with retention history, since the data itself doesn't
have reference to node IDs.

(I may sound definitive, but have been wrong before - I hope others will
correct me if thats the case)


>
> Thanks,
> Wes
>
>