Best practices for running Storm, HBase, Kafka (regading Zookeeper cluster)

[Spico Florin <sp...@gmail.com>]

Hello!
 I would like to know how it is best to run the three systems regarding the
Zookeeper cluster usage:
 1. separately per each system (ZK cluster/Storm, Zk Cluster/HBase,
Zk/Hbase)
2. single cluster for all of them (Storm,HBase,Kafka->single ZK cluster)

In my opinion the first one is the best option (since each system uses
differently the ZK and we could have performance and availability issues).

I look forward forward for your advises.
  Regards,
 Florin

Re: Best practices for running Storm, HBase, Kafka (regading Zookeeper cluster)

[Peyman Mohajerian <mo...@gmail.com>]

Option 1 clearly is more reliable but some choose option 2 to reduce the
overhead of having multiple ZK cluster from resource and management point
of view. I've seen both cases and more than anything else depends on you
workload on each of the components. For example Kafka and Storm may run on
the same ZK cluster and just split up HBase if the load on HBase is way
more. Another practice I have seen is increase the ZK instance from default
of 3 to 5 nodes to handle additional load, I think that's more common.

On Wed, Mar 2, 2016 at 2:37 AM, Spico Florin <sp...@gmail.com> wrote:

> Hello!
>  I would like to know how it is best to run the three systems regarding
> the Zookeeper cluster usage:
>  1. separately per each system (ZK cluster/Storm, Zk Cluster/HBase,
> Zk/Hbase)
> 2. single cluster for all of them (Storm,HBase,Kafka->single ZK cluster)
>
> In my opinion the first one is the best option (since each system uses
> differently the ZK and we could have performance and availability issues).
>
> I look forward forward for your advises.
>   Regards,
>  Florin
>
>
>