You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Bennett, Conrad" <Co...@VerizonWireless.com.INVALID> on 2017/05/25 05:25:31 UTC
Kafka Configuration Question
Hello,
I'm hoping someone could provide me with some assistance please. I am in the process of attempting to standing up a Kafka cluster and I have 7 nodes all of which has kafka and zookeeper installed. I have attached my server.properties file to verify whether or not I have anything misconfigured but each time I try to start the kafka service it fails with the error timed out connecting to zookeeper but the zookeeper process is up and running. Also during my research I read in order to achieve better performance separate drives for kafka data should be configure, but in the configuration file I didn't understand where exactly that should be configure. Any assistance would be greatly appreciated. Thanks in advance
kafka: { version: 0.10.1.1 }
zkper: { version: 3.4.9 }
Conrad Bennett Jr.
Re: Kafka Configuration Question
Posted by Sandeep Nemuri <nh...@gmail.com>.
Trace of the error could be helpful to see what is happening.
On Thu, May 25, 2017 at 6:50 PM, Raghav <ra...@gmail.com> wrote:
> Looks like you missed attachment of your server.properties file.
>
> On Wed, May 24, 2017 at 10:25 PM, Bennett, Conrad <
> Conrad.Bennett@verizonwireless.com.invalid> wrote:
>
> > Hello,
> >
> > I’m hoping someone could provide me with some assistance please. I am in
> > the process of attempting to standing up a Kafka cluster and I have 7
> nodes
> > all of which has kafka and zookeeper installed. I have attached my
> > server.properties file to verify whether or not I have anything
> > misconfigured but each time I try to start the kafka service it fails
> with
> > the error timed out connecting to zookeeper but the zookeeper process is
> up
> > and running. Also during my research I read in order to achieve better
> > performance separate drives for kafka data should be configure, but in
> the
> > configuration file I didn’t understand where exactly that should be
> > configure. Any assistance would be greatly appreciated. Thanks in advance
> >
> > kafka: { version: 0.10.1.1 }
> >
> > zkper: { version: 3.4.9 }
> >
> > Conrad Bennett Jr.
> >
> >
>
>
> --
> Raghav
>
--
* Regards*
* Sandeep Nemuri*
Re: Kafka Configuration Question
Posted by Raghav <ra...@gmail.com>.
Looks like you missed attachment of your server.properties file.
On Wed, May 24, 2017 at 10:25 PM, Bennett, Conrad <
Conrad.Bennett@verizonwireless.com.invalid> wrote:
> Hello,
>
> I’m hoping someone could provide me with some assistance please. I am in
> the process of attempting to standing up a Kafka cluster and I have 7 nodes
> all of which has kafka and zookeeper installed. I have attached my
> server.properties file to verify whether or not I have anything
> misconfigured but each time I try to start the kafka service it fails with
> the error timed out connecting to zookeeper but the zookeeper process is up
> and running. Also during my research I read in order to achieve better
> performance separate drives for kafka data should be configure, but in the
> configuration file I didn’t understand where exactly that should be
> configure. Any assistance would be greatly appreciated. Thanks in advance
>
> kafka: { version: 0.10.1.1 }
>
> zkper: { version: 3.4.9 }
>
> Conrad Bennett Jr.
>
>
--
Raghav
Re: [E] Re: Kafka Configuration Question
Posted by Hans Jespersen <ha...@confluent.io>.
Not sure why you would need 7 zookeepers, 3 or 5 is more common but since you have setup 7 then zookeeper will not be active unless a “quorum” of the zookeepers are up and running. That means you need to have 4 or more zookeepers up and running first, and then start your kafka brokers.
Also, you only included one server.properties file. You will need to give each of the 7 brokers each a unique broker.id and host.name. In fact host.name is deprecated and it looks like you might be incorrectly setting them all to “0.0.0.0” when you should instead be setting listeners to PLAINTEXT://0.0.0.0:9092 <plaintext://0.0.0.0:9092> to get the broker to listen to all interfaces.
From https://kafka.apache.org/documentation/ <https://kafka.apache.org/documentation/>
host.name DEPRECATED: only used when `listeners` is not set. Use `listeners` instead. hostname of broker. If this is set, it will only bind to this address. If this is not set, it will bind to all interfaces
listeners Listener List - Comma-separated list of URIs we will listen on and the listener names. If the listener name is not a security protocol, listener.security.protocol.map must also be set. Specify hostname as 0.0.0.0 to bind to all interfaces. Leave hostname empty to bind to default interface. Examples of legal listener lists: PLAINTEXT://myhost:9092,SSL://:9091 CLIENT://0.0.0.0:9092,REPLICATION://localhost:
Lastly you have set advertised.listeners=PLAINTEXT://kafka-1.site-s2.mon.com:9092 <plaintext://kafka-1.site-s2.mon.com:9092> which would override the value of listeners and host.name and force the broker to only listen on port 9092 on the IP address for kafka-1.site-s2.mon.com <http://kafka-1.site-s2.mon.com/>. Is this what you really want to do?
advertised.listeners Listeners to publish to ZooKeeper for clients to use, if different than the listeners above. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for `listeners` will be used.
If you set all this up properly and still get an error then please include the errors from both zookeeper and kafka brokers to help in debugging further
-hans
--
/**
* Hans Jespersen, Principal Systems Engineer, Confluent Inc.
* hans@confluent.io (650)924-2670
*/
> On May 29, 2017, at 1:33 AM, Bennett, Conrad <Co...@VerizonWireless.com.INVALID> wrote:
>
> Hello – anyone had a chance to take a look at this for me please? My cluster still isn’t up successfully. Thanks in advance
> Conrad Bennett Jr.
> From: Conrad Bennett <conrad.bennett@verizonwireless.com <ma...@verizonwireless.com>>
> Date: Friday, May 26, 2017 at 12:48 PM
> To: SenthilKumar K <senthilec566@gmail.com <ma...@gmail.com>>
> Subject: Re: [E] Re: Kafka Configuration Question
>
> Hi SenthilKumar,
>
> Did you get a chance to review the configuration? I am still unsuccessful with getting the cluster up.
> Conrad Bennett Jr.
> From: Conrad Bennett <conrad.bennett@verizonwireless.com <ma...@verizonwireless.com>>
> Date: Thursday, May 25, 2017 at 10:16 AM
> To: SenthilKumar K <senthilec566@gmail.com <ma...@gmail.com>>
> Subject: Re: [E] Re: Kafka Configuration Question
>
> Here is the attachment.
> Conrad Bennett Jr.
>
> From: SenthilKumar K <senthilec566@gmail.com <ma...@gmail.com>>
> Date: Thursday, May 25, 2017 at 3:09 AM
> To: Conrad Bennett <conrad.bennett@verizonwireless.com <ma...@verizonwireless.com>>
> Subject: [E] Re: Kafka Configuration Question
>
> Hi , Seems to be you missed attachment ..
>
> On Thu, May 25, 2017 at 10:55 AM, Bennett, Conrad <Conrad.Bennett@verizonwireless.com.invalid <ma...@verizonwireless.com.invalid>> wrote:
>> Hello,
>>
>> I’m hoping someone could provide me with some assistance please. I am in the process of attempting to standing up a Kafka cluster and I have 7 nodes all of which has kafka and zookeeper installed. I have attached my server.properties file to verify whether or not I have anything misconfigured but each time I try to start the kafka service it fails with the error timed out connecting to zookeeper but the zookeeper process is up and running. Also during my research I read in order to achieve better performance separate drives for kafka data should be configure, but in the configuration file I didn’t understand where exactly that should be configure. Any assistance would be greatly appreciated. Thanks in advance
>>
>> kafka: { version: 0.10.1.1 }
>>
>> zkper: { version: 3.4.9 }
>>
>> Conrad Bennett Jr.
>
> <server.properties.zip>
Re: [E] Re: Kafka Configuration Question
Posted by "Bennett, Conrad" <Co...@VerizonWireless.com.INVALID>.
Hello - anyone had a chance to take a look at this for me please? My cluster still isn't up successfully. Thanks in advance
Conrad Bennett Jr.
From: Conrad Bennett <co...@verizonwireless.com>>
Date: Friday, May 26, 2017 at 12:48 PM
To: SenthilKumar K <se...@gmail.com>>
Subject: Re: [E] Re: Kafka Configuration Question
Hi SenthilKumar,
Did you get a chance to review the configuration? I am still unsuccessful with getting the cluster up.
Conrad Bennett Jr.
From: Conrad Bennett <co...@verizonwireless.com>>
Date: Thursday, May 25, 2017 at 10:16 AM
To: SenthilKumar K <se...@gmail.com>>
Subject: Re: [E] Re: Kafka Configuration Question
Here is the attachment.
Conrad Bennett Jr.
From: SenthilKumar K <se...@gmail.com>>
Date: Thursday, May 25, 2017 at 3:09 AM
To: Conrad Bennett <co...@verizonwireless.com>>
Subject: [E] Re: Kafka Configuration Question
Hi , Seems to be you missed attachment ..
On Thu, May 25, 2017 at 10:55 AM, Bennett, Conrad <Co...@verizonwireless.com.invalid>> wrote:
Hello,
I'm hoping someone could provide me with some assistance please. I am in the process of attempting to standing up a Kafka cluster and I have 7 nodes all of which has kafka and zookeeper installed. I have attached my server.properties file to verify whether or not I have anything misconfigured but each time I try to start the kafka service it fails with the error timed out connecting to zookeeper but the zookeeper process is up and running. Also during my research I read in order to achieve better performance separate drives for kafka data should be configure, but in the configuration file I didn't understand where exactly that should be configure. Any assistance would be greatly appreciated. Thanks in advance
kafka: { version: 0.10.1.1 }
zkper: { version: 3.4.9 }
Conrad Bennett Jr.