Removing and re-adding broker

[Pankaj Gupta <pa...@gmail.com>]

Hi,

I have been reading about the Kafka design and I'm really impressed. I really like how you use OS page caching and message offsets as ids to achieve simplicity. I also like that you use disks very well and hence are able to support the pull model, which simplifies things so much. I have a couple questions.

If a Broker machine goes down, is restarted and hostname of the machine changes, in this case would the logs on the machine become available or would they be lost. I would really appreciate if you could provide some details on how this would work.

I see that a consumer going down triggers a rebalance and the paper mentions that the rebalance happens pretty quickly. If we have a fickle consumer that for some reason starts up and crashes, again and again, would that block the entire Kafka system?


Thanks in Advance,
Pankaj

Re: Removing and re-adding broker

[Joel Koshy <jj...@gmail.com>]

>
> If a Broker machine goes down, is restarted and hostname of the machine
> changes, in this case would the logs on the machine become available or
> would they be lost. I would really appreciate if you could provide some
> details on how this would work.


The logs will still be available.


>
> I see that a consumer going down triggers a rebalance and the paper
> mentions that the rebalance happens pretty quickly. If we have a fickle
> consumer that for some reason starts up and crashes, again and again, would
> that block the entire Kafka system?
>
>
Only that consumer group will be affected. Other consumer groups and
producers should be fine.

Thanks,

Joel