You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Janitha Jayaweera <ja...@hotmail.com> on 2019/04/30 17:52:35 UTC

My IDE which is sitting outside of the kube cluster creates a producer that attempts to connect to kafka using the cluster dns name of the headless service

Hi!

The Kafka chart being used is here https://github.com/bitnami/charts/tree/master/bitnami/kafka

| chart | application |
| kafka-1.10.1 | 2.2.0 |


$ kubectl get svc -n kafka

NAME                               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
kafka-janitha                      ClusterIP   10.103.152.8     <none>        9092/TCP                     5d19h
kafka-janitha-headless             ClusterIP   None             <none>        9092/TCP                     5d19h
kafka-janitha-zookeeper            ClusterIP   10.108.191.161   <none>        2181/TCP,2888/TCP,3888/TCP   5d19h
kafka-janitha-zookeeper-headless   ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   5d19h

I am then port-forwarding like so

kubectl port-forward --namespace postgresql svc/postgres-janitha-postgresql 5432:5432 --address 0.0.0.0 &
kubectl port-forward --namespace kafka svc/kafka-janitha 9092:9092 --address 0.0.0.0 &
kubectl port-forward --namespace activemq svc/activemq-tideworks-activemq 61616:61616 --address 0.0.0.0 &

I have my application properties set like so where the ip is the private ip of the node (master). My IDE is running outside the cluster.

tc.db.jdbc.url=jdbc:postgresql://172.31.31.192:5432/tcb_configuration
#
activemq.base.url=tcp://172.31.31.192:61616?wireFormat.maxInactivityDuration=-1
#
kafka.bootstrap.servers=172.31.31.192:9092

The application log states the following

17:36:45.617 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.618 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.667 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.668 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.717 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.718 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.768 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.768 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.818 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.818 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.868 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.869 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.918 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Give up sending metadata request since no node is available
17:36:45.919 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.969 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Initialize connection to node kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local:9092 (id: 1001 rack: null) for sending metadata request
17:36:45.969 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Initiating connection to node kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local:9092 (id: 1001 rack: null)
17:36:45.969 [kafka-producer-network-thread | producer-5] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-5] Give up sending metadata request since no node is available
17:36:45.969 [kafka-producer-network-thread | producer-3] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] Error connecting to node kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local:9092 (id: 1001 rack: null)
java.io.IOException: Can't resolve address: kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local:9092
               at org.apache.kafka.common.network.Selector.doConnect(Selector.java:223) ~[kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.common.network.Selector.connect(Selector.java:202) ~[kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:793) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.NetworkClient.access$700(NetworkClient.java:62) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:944) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:848) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:458) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) [kafka-clients-1.0.2.jar:?]
               at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) [kafka-clients-1.0.2.jar:?]
               at java.lang.Thread.run(Thread.java:844) [?:?]
Caused by: java.nio.channels.UnresolvedAddressException
               at sun.nio.ch.Net.checkAddress(Net.java:112) ~[?:?]
               at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) ~[?:?]
               at org.apache.kafka.common.network.Selector.doConnect(Selector.java:221) ~[kafka-clients-1.0.2.jar:?]
               ... 9 more

I think this is because my IDE which is sitting outside of the kube cluster is creates a producer that attempts to connect to kafka using the cluster dns name of the headless service.

I haven't finished checking whether the application will run inside the kube cluster, but likely yes given that the following works

$ kubectl create -f https://k8s.io/examples/admin/dns/busybox.yaml

$ kubectl -n default exec -ti busybox -- nslookup kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kafka-janitha-0.kafka-janitha-headless.kafka.svc.cluster.local
Address 1: 192.168.150.22 192-168-150-22.kafka-janitha.kafka.svc.cluster.local

Is sort of setup possible? What am I missing / what do I not know?

Thanks

Janitha