You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by sat <sa...@gmail.com> on 2016/10/17 04:17:31 UTC
Kafka consumer cluster setup
Hi,
We are planning to run Kafka servers along with Zookeeper in 3 remote
machines for our cluster setup.
We have our application with Kafka Consumer and Publisher running in
another/separate 2 machines.
We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers
in Kafka Consumer/Publisher properties.
Since we are configuring 1 or 2 Kafka server IP address in bootstrap server
list, we want to know when one or both the servers configured as bootstrap
server crashes, can we still consume/publish messages to the other server
available in the cluster. If this is possible, could you please let us
know whether our application opens a socket with all 3 machines in the
server.
Also please let us know if it is fine to configure/provide only 1 machine
out of 3 as bootstrap server.
Please let me know if in case you need additional information. Thanks in
advance.
Thanks and Regards
A.SathishKumar
Re: Kafka consumer cluster setup
Posted by Ian Wrigley <ia...@confluent.io>.
Hi
The client initially connects to the first Broker specified in bootstrap.servers (if that’s not available, it’ll connect to the next one on the list, and so on). When it does so, it is then given information about all the Brokers in the cluster, and it connects to all of them. Strictly speaking, then, you only need to specify one Broker in the bootstrap.servers list, but if that had crashed when the client tried to connect, the client would throw an error after the connection attempt timed out, which is why you’d typically specify two or three in that list. Certainly, though, that list isn’t a list of the only Brokers that the client will talk to as it’s running.
Ian.
---
Ian Wrigley
Director, Education Services
Confluent, Inc
> On Oct 16, 2016, at 11:17 PM, sat <sa...@gmail.com> wrote:
>
> Hi,
>
> We are planning to run Kafka servers along with Zookeeper in 3 remote
> machines for our cluster setup.
>
> We have our application with Kafka Consumer and Publisher running in
> another/separate 2 machines.
>
> We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers
> in Kafka Consumer/Publisher properties.
>
> Since we are configuring 1 or 2 Kafka server IP address in bootstrap server
> list, we want to know when one or both the servers configured as bootstrap
> server crashes, can we still consume/publish messages to the other server
> available in the cluster. If this is possible, could you please let us
> know whether our application opens a socket with all 3 machines in the
> server.
>
> Also please let us know if it is fine to configure/provide only 1 machine
> out of 3 as bootstrap server.
>
> Please let me know if in case you need additional information. Thanks in
> advance.
>
>
>
> Thanks and Regards
> A.SathishKumar
Re: Kafka consumer cluster setup
Posted by sat <sa...@gmail.com>.
Hi Ian,
Thanks for the detailed explanation.
Thanks and Regards
A.SathishKumar
> Hi
> The client initially connects to the first Broker specified in bootstrap.servers (if that’s
> not available, it’ll connect to the next one on the list, and so on). When it does so, it
> is then given information about all the Brokers in the cluster, and it connects to all of
> them. Strictly speaking, then, you only need to specify one Broker in the bootstrap.servers
> list, but if that had crashed when the client tried to connect, the client would throw an
> error after the connection attempt timed out, which is why you’d typically specify two or
> three in that list. Certainly, though, that list isn’t a list of the only Brokers that the
> client will talk to as it’s running.
Ian.
---
Ian Wrigley
Director, Education Services
Confluent, Inc
> On Oct 16, 2016, at 11:17 PM, sat <sa...@gmail.com> wrote:
>
> Hi,
>
> We are planning to run Kafka servers along with Zookeeper in 3 remote
> machines for our cluster setup.
>
> We have our application with Kafka Consumer and Publisher running in
> another/separate 2 machines.
>
> We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers
> in Kafka Consumer/Publisher properties.
>
> Since we are configuring 1 or 2 Kafka server IP address in bootstrap server
> list, we want to know when one or both the servers configured as bootstrap
> server crashes, can we still consume/publish messages to the other server
> available in the cluster. If this is possible, could you please let us
> know whether our application opens a socket with all 3 machines in the
> server.
>
> Also please let us know if it is fine to configure/provide only 1 machine
> out of 3 as bootstrap server.
>
> Please let me know if in case you need additional information. Thanks in
> advance.
>
>
>
> Thanks and Regards
> A.SathishKumar
On Sun, Oct 16, 2016 at 9:17 PM, sat <sa...@gmail.com> wrote:
> Hi,
>
> We are planning to run Kafka servers along with Zookeeper in 3 remote
> machines for our cluster setup.
>
> We have our application with Kafka Consumer and Publisher running in
> another/separate 2 machines.
>
> We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers
> in Kafka Consumer/Publisher properties.
>
> Since we are configuring 1 or 2 Kafka server IP address in bootstrap
> server list, we want to know when one or both the servers configured as
> bootstrap server crashes, can we still consume/publish messages to the
> other server available in the cluster. If this is possible, could you
> please let us know whether our application opens a socket with all 3
> machines in the server.
>
> Also please let us know if it is fine to configure/provide only 1 machine
> out of 3 as bootstrap server.
>
> Please let me know if in case you need additional information. Thanks in
> advance.
>
>
>
> Thanks and Regards
> A.SathishKumar
>
>
--
A.SathishKumar
044-24735023