You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/26 07:50:36 UTC
[GitHub] [pulsar] mianos opened a new issue #8373: Stock helm install, standard python client example, proxy does not proxy, just discovers and redirects
mianos opened a new issue #8373:
URL: https://github.com/apache/pulsar/issues/8373
**Description**
Helm installed Pulsar proxy does not seem to proxy.
No TLS.
Maybe my misunderstanding the clear documentation.
I assumed that the Pulsar proxy should proxy all Pulsar traffic into Kubernetes according to the documentation.
The Pulsar brokers should not need external addresses, as configured by default in the helm chart.
**To Reproduce**
Install pulsar via helm (with some persistent volumes as needed already set up):
Helm values are basically a one node k8s cluster with some local persistent storage.
No grafana, no prometheus, no affinity.
https://gist.github.com/mianos/1d84bfe7f580e6a3917ed8506c42e96e
Installing:
```
helm -n pulsar install --set initialize=true --values=values-as-above.yaml pulsar apache/pulsar
```
Wait a while.
```
NAME READY STATUS RESTARTS AGE
pulsar-bookie-0 1/1 Running 0 25m
pulsar-bookie-init-jlrvv 0/1 Completed 0 25m
pulsar-broker-0 1/1 Running 0 25m
pulsar-proxy-0 1/1 Running 0 25m
pulsar-pulsar-init-5glmm 0/1 Completed 0 25m
pulsar-pulsar-manager-76c5cfb97f-nknr5 1/1 Running 0 25m
pulsar-toolset-0 1/1 Running 0 25m
pulsar-zookeeper-0 1/1 Running 0 25m
```
I also patched the proxy load balancer external IP to give me some network connectivity
```
kubectl --namespace pulsar patch svc pulsar-proxy -p '{"spec": {"type": "LoadBalancer", "externalIPs":["131.84.1.232"]}}'
```
Create a basic python client:
https://gist.github.com/mianos/3d3541408902a7381c42a2740ac08d76
pip installing pulsar-client==2.6.1 to a local venv
```
kubectl -n pulsar get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
pulsar-bookie ClusterIP None <none> 3181/TCP,8000/TCP 26m
pulsar-broker ClusterIP None <none> 8080/TCP,6650/TCP 26m
pulsar-proxy LoadBalancer 10.102.14.255 131.84.1.232 80:30172/TCP,6650:32728/TCP 26m
pulsar-pulsar-manager LoadBalancer 10.107.242.134 131.84.1.232 9527:32067/TCP 26m
pulsar-toolset ClusterIP None <none> <none> 26m
pulsar-zookeeper ClusterIP None <none> 2888/TCP,3888/TCP,2181/TCP 26m
```
**Run**
Running the python app above:
```
2020-10-26 07:36:20.331 INFO [140172044347136] HTTPLookupService:237 | Curl Lookup Request sent for http://131.84.1.232/admin/v2/persistent/public/default/test_topic/partitions
2020-10-26 07:36:20.337 INFO [140172044347136] HTTPLookupService:251 | Response received for url http://131.84.1.232/admin/v2/persistent/public/default/test_topic/partitions code 200
2020-10-26 07:36:20.337 INFO [140172044347136] HTTPLookupService:295 | parsePartitionData = { LookupDataResult [brokerUrl_ = ] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2020-10-26 07:36:20.337 INFO [140172044347136] HandlerBase:53 | [persistent://public/default/test_topic, ] Getting connection from pool
2020-10-26 07:36:20.337 INFO [140172044347136] HTTPLookupService:237 | Curl Lookup Request sent for http://131.84.1.232/lookup/v2/topic/persistent/public/default/test_topic
2020-10-26 07:36:20.339 INFO [140172044347136] HTTPLookupService:251 | Response received for url http://131.84.1.232/lookup/v2/topic/persistent/public/default/test_topic code 200
2020-10-26 07:36:20.339 INFO [140172044347136] HTTPLookupService:330 | parseLookupData = { LookupDataResult [brokerUrl_ = pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650] [brokerUrlTls_ = ] [partitions = 0] [authoritative = 0] [redirect = 0] proxyThroughServiceUrl = 0] }
2020-10-26 07:36:20.340 INFO [140172044347136] ConnectionPool:85 | Created connection for pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650
2020-10-26 07:36:20.344 ERROR [140171961956096] ClientConnection:469 | [<none> -> pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650] Resolve error: asio.netdb:2 : Host not found (non-authoritative), try again later
2020-10-26 07:36:20.344 INFO [140171961956096] ClientConnection:1387 | [<none> -> pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650] Connection closed
2020-10-26 07:36:20.344 INFO [140171961956096] ProducerImpl:483 | Producer - [persistent://public/default/test_topic, ] , [batching = off]
2020-10-26 07:36:20.344 INFO [140171961956096] ClientConnection:238 | [<none> -> pulsar://pulsar-broker-0.pulsar-broker.pulsar.svc.cluster.local:6650] Destroyed connection
Traceback (most recent call last):
File "ptest.py", line 21, in <module>
producer = client.create_producer(opts.topic)
File "/home/rfo/wa/mcs-pulsar-companion/py/venv/lib/python3.8/site-packages/pulsar/__init__.py", line 525, in create_producer
p._producer = self._client.create_producer(topic, conf)
Exception: Pulsar error: ConnectError
```
**Expected behavior**
In the first instance, nothing should be displayed and the messages should be published into pulsar, worst case complains about missing topic or permission.
Note that asio.netdb: is a boost error, so this came from the C++ client underneath the python API
kubectl version
```Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:41:49Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
```
```
helm -n pulsar list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
pulsar pulsar 1 2020-10-26 07:07:37.884071269 +0000 UTC deployed pulsar-2.6.0 1.0
```
I also have a non helm installed pulsar without the proxy on some machine here. I'll give my python app a run on that just to verify it works.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mianos closed issue #8373: Stock helm install, standard python client example, proxy does not proxy, just discovers and redirects
Posted by GitBox <gi...@apache.org>.
mianos closed issue #8373:
URL: https://github.com/apache/pulsar/issues/8373
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mianos commented on issue #8373: Stock helm install, standard python client example, proxy does not proxy, just discovers and redirects
Posted by GitBox <gi...@apache.org>.
mianos commented on issue #8373:
URL: https://github.com/apache/pulsar/issues/8373#issuecomment-716396539
My mistake! I just knew it.
I had the host as: pulsar_host = "pulsar_host = "**http**://131.84.1.232:6650"
not "**pulsar**://131.84.1.232:6650"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org