You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/04/06 09:11:07 UTC
Slack digest for #general - 2018-04-06
2018-04-05 10:00:24 UTC - Colm O'Reilly: @Colm O'Reilly has joined the channel
----
2018-04-05 22:47:02 UTC - Yuwei Jiang: Hi guys, I’m fairly new to Pulsar, I’m playing with having it installed on a Kubernetes cluster. I followed the documentation and have all the components running on K8s correctly, I have the proxy running as `NodePort` and exposing port 6650 through port 30002.
I encountered an issue when running a subscriber via python client to connect to the service url `pulsar://<K8s node ip>:30002`, from outside of K8s. I received the following error:
```
15:12:38.146 [0x700003fd0000] ERROR /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientImpl.cc(240) - Error Checking/Getting Partition Metadata while Subscribing- 5
```
I thought I can gain some insight on the issue by asking the question here. Thank you!
----
2018-04-05 22:50:58 UTC - Matteo Merli: @Yuwei Jiang When connecting from outside the K8S cluster, the problem is to have direct connectivity to the Pulsar broker pods. There are few ways to solve that :
1. Configure broker to use host network so they will bind on an IP that is reachable from outside K8S. This also involves using the host IP as the `advertisedAddress` for the brokers
2. Run an instance of Pulsar proxy. The proxy is completely stateless and can be exposed through load-balancer, NodePort, etc..
----
2018-04-05 23:00:43 UTC - Yuwei Jiang: Hi @Matteo Merli, thanks for the prompt response! I didn’t encounter the issue yesterday, meaning I was able to create topic and subscribe from /publish to it, with python client. I noticed the error message and wondering if there something special about it. Is it checking the metadata stored in zookeeper?
----
2018-04-05 23:02:30 UTC - Matteo Merli: the client doesn’t connect directly to zookeeper, though, in absence of the Pulsar proxy, the client needs to have direct connectivity to brokers
----
2018-04-05 23:03:00 UTC - Matteo Merli: can you share the complete logs with the error?
----
2018-04-05 23:06:58 UTC - Yuwei Jiang: Yes, I have proxy with 3 replicas running. Below is the output when I run my subscriber:
```
15:12:38.033 [0x7fffb4790380] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created connection for <pulsar://172.29.203.188:30002>
15:12:38.146 [0x700003fd0000] ERROR /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(323) - [<none> -> <pulsar://172.29.203.188:30002>] Failed to establish connection: Connection refused
15:12:38.146 [0x700003fd0000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(1126) - [<none> -> <pulsar://172.29.203.188:30002>] Connection closed
15:12:38.146 [0x700003fd0000] ERROR /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientImpl.cc(240) - Error Checking/Getting Partition Metadata while Subscribing- 5
15:12:38.146 [0x700003fd0000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(189) - [<none> -> <pulsar://172.29.203.188:30002>] Destroyed connection
Traceback (most recent call last):
File "c8streams_consumer1.py", line 36, in <module>
consumer = client.subscribe(TOPIC, SUBSCRIPTION)
File "/usr/local/lib/python3.6/site-packages/pulsar.py", line 427, in subscribe
c._consumer = self._client.subscribe(topic, subscription_name, conf)
Exception: Pulsar error: ConnectError
```
----
2018-04-05 23:08:27 UTC - Yuwei Jiang: Just ran again and I got the following error:
```
16:06:27.639 [0x7fffb4790380] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created connection for <pulsar://172.29.203.188:30002>
16:06:27.665 [0x700001a85000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(279) - [172.29.219.211:55393 -> 172.29.203.188:30002] Connected to broker
16:06:27.693 [0x700001a85000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/HandlerBase.cc(53) - [<persistent://mm/global/c8/fake>, test-sub, 0] Getting connection from pool
16:06:27.719 [0x700001a85000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created connection for <pulsar://10.243.65.70:6650>
16:06:28.738 [0x700001a85000] ERROR /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(323) - [<none> -> <pulsar://172.29.203.188:30002>] Failed to establish connection: Connection refused
16:06:28.738 [0x700001a85000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(1126) - [<none> -> <pulsar://172.29.203.188:30002>] Connection closed
16:06:28.738 [0x700001a85000] INFO /Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(189) - [<none> -> <pulsar://172.29.203.188:30002>] Destroyed connection
Traceback (most recent call last):
File "c8streams_consumer1.py", line 36, in <module>
consumer = client.subscribe(TOPIC, SUBSCRIPTION)
File "/usr/local/lib/python3.6/site-packages/pulsar.py", line 427, in subscribe
c._consumer = self._client.subscribe(topic, subscription_name, conf)
Exception: Pulsar error: ConnectError
```
----
2018-04-05 23:21:57 UTC - Sijie Guo: @Yuwei Jiang can you try “telnet 172.29.203.188 30002”?
----
2018-04-05 23:27:07 UTC - Matteo Merli: > Created connection for <pulsar://10.243.65.70:6650>
That’s what I was mentioning, the client is directed to connect to the broker pod, at the pod IP which is internal to K8S
----
2018-04-05 23:27:49 UTC - Matteo Merli: take a look at <https://github.com/apache/incubator-pulsar/blob/master/deployment/kubernetes/generic/proxy.yaml>
----
2018-04-05 23:28:09 UTC - Matteo Merli: to deploy a proxy service in the K8S cluster and use that as your endpoint
----
2018-04-05 23:28:16 UTC - Matteo Merli: for the NodePort
----
2018-04-05 23:35:23 UTC - Yuwei Jiang: Thank you guys for the direction! Looks like something with my proxy configuration; let me look into it.
----