restoring broker node / zookeeper question

[Ran RanUser <ra...@gmail.com>]

I have two questions:

1.

We plan to initially run Kafka without partitioning or replication, simply
having two single node Kafka (broker) servers in two geographically diverse
locations.  Using periodic backups/snapshots across locations for disaster
recovery.

Our producer / consumer applications will process to/from each location.

For such configurations, is it possible to run the Kafka broker without
Zookeeper?  If not, any plans for such a simplified configuration?  We'd
like to remove the extra app from the stack (Zookeeper) since it's not
being utilized.

2.

If we need to restore a Kafka node (simplified design above), I was hoping
Kafka and Zookeeper could bootstrap itself as long as we can restore a copy
of the /tmp/kafka-logs data. This dir contains the topics, log data, and
high watermark.

Unfortunately, it seems without the Zookeeper data, we cannot restore the
node.

Is there a way to restore a Kafka node if the Zookeeper data cannot be
restored?  What are the specific requirements to restore a Kafka broker
from backups, if the /tmp/zookeeper must also be restored, are there any
inconsistent state issues (i.e. backup of /tmp/kafka-logs and
/tmp/zookeeper are not synchronized)?


Thank you!

Re: restoring broker node / zookeeper question

[Jun Rao <ju...@gmail.com>]

In 0.8, Kafka broker can't be run w/o ZK. In 0.7, you can run a broker
without ZK. However, you won't able to run the high level consumer w/o ZK.

Thanks,

Jun


On Sat, Jan 11, 2014 at 6:52 PM, Ran RanUser <ra...@gmail.com> wrote:

> I have two questions:
>
> 1.
>
> We plan to initially run Kafka without partitioning or replication, simply
> having two single node Kafka (broker) servers in two geographically diverse
> locations.  Using periodic backups/snapshots across locations for disaster
> recovery.
>
> Our producer / consumer applications will process to/from each location.
>
> For such configurations, is it possible to run the Kafka broker without
> Zookeeper?  If not, any plans for such a simplified configuration?  We'd
> like to remove the extra app from the stack (Zookeeper) since it's not
> being utilized.
>
> 2.
>
> If we need to restore a Kafka node (simplified design above), I was hoping
> Kafka and Zookeeper could bootstrap itself as long as we can restore a copy
> of the /tmp/kafka-logs data. This dir contains the topics, log data, and
> high watermark.
>
> Unfortunately, it seems without the Zookeeper data, we cannot restore the
> node.
>
> Is there a way to restore a Kafka node if the Zookeeper data cannot be
> restored?  What are the specific requirements to restore a Kafka broker
> from backups, if the /tmp/zookeeper must also be restored, are there any
> inconsistent state issues (i.e. backup of /tmp/kafka-logs and
> /tmp/zookeeper are not synchronized)?
>
>
> Thank you!
>