You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by R Krishna <kr...@gmail.com> on 2016/05/14 02:43:36 UTC
Suggestions with hostnames in a Kafka cluster
As a newbie, just setup my first Kafka 3 node, each in its own host cluster
and its own ZK. Everything went fine, I could see three brokers registered
in all three ZK /brokers/ids until I created a topic with this exception:
~/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181
--replication-factor 3 --partitions 1 --topic kafkatest
[2016-05-14 02:22:40,003] INFO Partition [kafkatest,0] on broker 1: No
checkpointed highwatermark is found for partition [kafkatest,0]
(kafka.cluster.Partion)
[2016-05-14 02:22:40,007] INFO [ReplicaFetcherManager on broker 1] Removed
fetcher for partitions [kafkatest,0] (kafka.server.ReplicaFetcherManager)
[2016-05-14 02:22:40,010] INFO Truncating log kafkatest-0 to offset 0.
(kafka.log.Log)
[2016-05-14 02:22:40,035] INFO [ReplicaFetcherThread-0-2], Starting
(kafka.server.ReplicaFetcherThread)
[2016-05-14 02:22:40,045] INFO [ReplicaFetcherManager on broker 1] Added
fetcher for partitions List([[kafkatest,0], initOffset 0 to broker
BrokerEndPoin2,localhost,9094)] ) (kafka.server.ReplicaFetcherManager)
[2016-05-14 02:22:40,050] WARN [ReplicaFetcherThread-0-2], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@4983f43d. Possible cause:
java.iIOException: Connection to Node(2, localhost, 9094) failed
(kafka.server.ReplicaFetcherThread)
[2016-05-14 02:22:42,054] WARN [ReplicaFetcherThread-0-2], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@256e458c. Possible cause:
java.iIOException: Connection to Node(2, localhost, 9094) failed
(kafka.server.ReplicaFetcherThread)
[2016-05-14 02:22:44,055] WARN [ReplicaFetcherThread-0-2], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@724f83b4. Possible cause:
java.i
Q.) Obviously, it could not recognize localhost, 9094, so what are some
best practices and where should I include IP addresses or resolvable
hostnames below? Without creating a topic, how do I know that the Kafka
cluster has recognized all three brokers and is stable?
Configuration File::::::::::::::::::::::::::::::
listeners=PLAINTEXT://localhost:9095
# The port the socket server listens on
# port unique 9093, 9094,...
port=9095
# Hostname the broker will bind to. If not set, the server will bind to all
interfaces
#host.name=localhost
# Hostname the broker will advertise to producers and consumers. If not
set, it uses the
# value for "host.name" if configured. Otherwise, it will use the value
returned from
# java.net.InetAddress.getCanonicalHostName().
#advertised.host.name=<hostname routable by clients>
# The port to publish to ZooKeeper for clients to use. If this is not set,
# it will publish the same port that the broker binds to.
#advertised.port=<port accessible by clients>
kafka@awo-p05-kafk03:~$ netstat -atun | grep LISTEN
tcp 0 0 0.0.0.0:8139 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::38303 :::* LISTEN
tcp6 0 0 :::2181 :::* LISTEN
tcp6 0 0 127.0.0.1:9095 :::* LISTEN
<<<<<<<<<<<
tcp6 0 0 :::2888 :::* LISTEN
tcp6 0 0 :::3888 :::* LISTEN
tcp6 0 0 :::33042 :::* LISTEN
tcp6 0 0 :::22 :::* LISTEN
-- Krishna