[GitHub] [pulsar] guillaume-braibant opened a new issue #4978: Pulsar - K8S : admin pod should use the broker service instead of proxy service to discover brokers

[GitBox <gi...@apache.org>]

guillaume-braibant opened a new issue #4978: Pulsar - K8S : admin pod should use the broker service instead of proxy service to discover brokers
URL: https://github.com/apache/pulsar/issues/4978
 
 
   ### Motivation
   
   Currently, the pulsar-admin pod uses the service provided in proxy.yml to discover broker pods and interact with them. Here is the part of the admin.yml script I am talking about :
   
   >         env:
   >          - name: webServiceUrl
   >            value: "http://proxy:8080/"
   >          - name: brokerServiceUrl
   >            value: "pulsar://proxy:6650/"
   >          - name: PULSAR_MEM
   >            value: "\"-Xms64m -Xmx128m\""
   
   The first problem is that you need to deploy the proxy service (proxy.yml) in order to make your pulsar-admin pod work. If you don't, your pulsar-admin pod will not be able to locate broker pods and, for example, perform the tests showed in the kubernetes deployment docs.
   
   The second problem is that you may not want to expose your broker to the outside world (= the whole internet for some managed kubernetes cluster like the ones from Digital Ocean).
   
   ### Proposed solution
   
   The idea is to use the headless broker service defined in broker.yml to perform broker pods discovery. Replace the previous part of the admin.yml script with this one :
   
   >        env:
   >          # Use the broker headless service to discover broker pods
   >          - name: webServiceUrl
   >            value: "http://broker:8080/"
   >          - name: brokerServiceUrl
   >            value: "pulsar://broker:6650/"
   >          - name: PULSAR_MEM
   >            value: "\"-Xms64m -Xmx128m\""
   
   This way, you can directly deploy the pulsar-admin pod after deploying the brokers and perform simple tests using the Pulsar Client CLI tool.
   
   ### Testing coverage
   
   The solution has been successfully tested on a fresh Digital Ocean K8S Cluster on which I deployed the zookeeper cluster, the bookies, the brokers and the admin pod (no monitoring stack yet). 
   
   The test consisted in producing messages at a 100 msg/s rate and consume them using the pulsar-perf command on the admin pod (as provided in the documentation).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services