You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/10/26 09:11:06 UTC
Slack digest for #general - 2020-10-26
2020-10-25 10:01:29 UTC - Jonas Dahl: @Jonas Dahl has joined the channel
----
2020-10-25 19:35:04 UTC - François Bélanger: @François Bélanger has joined the channel
----
2020-10-25 22:40:50 UTC - chris coovrey: @chris coovrey has joined the channel
----
2020-10-26 02:40:31 UTC - Jeremy Hanna: @Jeremy Hanna has joined the channel
----
2020-10-26 02:44:50 UTC - Jeremy Hanna: :wave: I was just wondering what the best way to contribute is - I created an asf jira issue but looks like the project uses a github workflow (as my issue is only issue number 8 :slightly_smiling_face: <https://issues.apache.org/jira/browse/PULSAR-8>)
----
2020-10-26 03:11:03 UTC - Rob Fowler: @Rob Fowler has joined the channel
----
2020-10-26 03:16:24 UTC - Rob Fowler: I just installed pulsar using the helm chart and set the external IP for the proxy as a load balancer but I seem to have the same issue
```020-10-26 14:06:30.364 INFO [139986756138816] Client:88 | Subscribing on Topic :test_topic
2020-10-26 14:06:30.365 INFO [139986516760320] HTTPLookupService:237 | Curl Lookup Request sent for <http://10.217.0.171:9080/admin/v2/persistent/public/default/test_topic/partitions>
2020-10-26 14:06:30.420 INFO [139986516760320] HTTPLookupService:251 | Response received for url <http://10.217.0.171:9080/admin/v2/persistent/public/default/test_topic/partitions> code 200
2020-10-26 14:06:30.420 INFO [139986516760320] HTTPLookupService:295 | parsePartitionData = { LookupDataResult [brokerUrl_ = ] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2020-10-26 14:06:30.420 INFO [139986516760320] HandlerBase:53 | [<persistent://public/default/test_topic>, ./ptest.py, 0] Getting connection from pool
2020-10-26 14:06:30.421 INFO [139986516760320] HTTPLookupService:237 | Curl Lookup Request sent for <http://10.217.0.171:9080/lookup/v2/topic/persistent/public/default/test_topic>
2020-10-26 14:06:30.434 INFO [139986516760320] HTTPLookupService:251 | Response received for url <http://10.217.0.171:9080/lookup/v2/topic/persistent/public/default/test_topic> code 200
2020-10-26 14:06:30.435 INFO [139986516760320] HTTPLookupService:330 | parseLookupData = { LookupDataResult [brokerUrl_ = <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2020-10-26 14:06:30.436 INFO [139986516760320] ConnectionPool:85 | Created connection for <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>
2020-10-26 14:06:30.449 ERROR [139986508367616] ClientConnection:469 | [<none> -> <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] Resolve error: asio.netdb:1 : Host not found (authoritative)
2020-10-26 14:06:30.449 INFO [139986508367616] ClientConnection:1387 | [<none> -> <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] Connection closed
2020-10-26 14:06:30.449 INFO [139986508367616] ClientConnection:238 | [<none> -> <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] Destroyed connection```
----
2020-10-26 03:16:42 UTC - Rob Fowler: and my service config is
```NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
pulsar-bookie ClusterIP None <none> 3181/TCP,8000/TCP 2d20h
pulsar-broker ClusterIP None <none> 8080/TCP,6650/TCP 2d20h
pulsar-proxy LoadBalancer 10.108.207.122 10.217.0.171 9080:31334/TCP,6650:30349/TCP 2d20h
pulsar-pulsar-manager LoadBalancer 10.108.137.132 10.217.0.171 9527:30749/TCP 2d20h
pulsar-toolset ClusterIP None <none> <none> 2d20h
pulsar-zookeeper ClusterIP None <none> 2888/TCP,3888/TCP,2181/TCP 2d20h```
----
2020-10-26 03:18:07 UTC - Rob Fowler: it seems, the python client talks to the proxy, then the proxy returns the broker address, then the client tries to connect to the broker on the address provided, obviously not possible as that is internal
----
2020-10-26 03:18:34 UTC - Rob Fowler: in my case, I have nothing in front of anything, it's just a local k8s cluster installed with helm
----
2020-10-26 03:22:02 UTC - Rob Fowler: using the python code from here: <https://github.com/apache/pulsar/blob/master/site2/docs/getting-started-docker.md>
----
2020-10-26 03:30:29 UTC - Rob Fowler: does the proxy config work with the helm chart? I have an almost line by line paste from the README and the python example but it seems the proxy is redirecting an external address to an internal broker.
----
2020-10-26 03:55:27 UTC - Rob Fowler: shouldn't the client be asking the proxy for the data? If I kubectl exec into the proxy machine, the host "pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local" is resolved so I don't think this is forwarded from the proxy process. (Also asio.netdb is a boost library, not java and the python seems to be a wrapper on the C++ client lib so that makes sense)
----
2020-10-26 03:57:21 UTC - Luke Stephenson: Is message Chunking supported with topic compaction? If I tombstone a message (publish with a null value), how do all of the chunks get cleaned up? Thanks
----
2020-10-26 03:59:40 UTC - Rob Fowler: proxyThroughServiceUrl = 0]
----
2020-10-26 04:00:15 UTC - Rob Fowler: ```2020-10-26 14:24:06.517 INFO [139910581896960] HTTPLookupService:251 | Response received for url <http://10.217.0.171:9080/lookup/v2/topic/persistent/public/default/test_topic-partition-0> code 200
2020-10-26 14:24:06.518 INFO [139910581896960] HTTPLookupService:330 | parseLookupData = { LookupDataResult [brokerUrl_ = <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2020-10-26 14:24:06.518 INFO [139910581896960] ConnectionPool:85 | Created connection for <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>
2020-10-26 14:24:06.528 ERROR [139910497429248] ClientConnection:469 | [<none> -> <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] Resolve error: asio.netdb:1 : Host not found (authoritative)
2020-10-26 14:24:06.528 INFO [139910497429248] ClientConnection:1387 | [<none> -> <pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650>] Connection closed```
----
2020-10-26 04:34:16 UTC - Rob Fowler: maybe I need to enable proxy ingress in k8s?
----
2020-10-26 08:46:33 UTC - Sijie Guo: We don’t use Jira. That one should be archived by the ASF infra team. @jia zhai is following up with that.
We use Github issues. You can create either a pull request or a Github issue to do that.
----
2020-10-26 08:47:40 UTC - Sijie Guo: What is your values files for the helm chart?
----