You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "youzipi (via GitHub)" <gi...@apache.org> on 2024/01/05 03:41:30 UTC
[I] how to direct connect to broker without proxy [pulsar-helm-chart]
youzipi opened a new issue, #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423
i would prefer not to use a proxy.
but i found broker does not have the ingress template.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1940550273
Related:
* [[PIP-338] Add default lookup listener and fix inconsistency with listener's usage between different protocols](https://github.com/apache/pulsar/pull/22039)
* [PIP-338 mailing list discussion](https://lists.apache.org/thread/0ythswmt6d0q10f1knctc7py0gh5s4rd)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1900179511
It would make sense to have a load balancer for the broker service that is used for lookups since the binary protocol is more efficient than using the REST API for lookups. The individual brokers need to be addressable directly and solving that requires a solution. I'd like to see an experiment for the nodeport + advertisedListeners solution. I guess that would be feasible in cloud managed k8s environments where it is possible to expose a k8s node with a routable address that the client could access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1895605385
Ingress probably wouldn't make sense for Pulsar brokers, at least for the binary protocol. For the Pulsar Admin API that would be a feasible approach.
You could use k8s node ports and Pulsar's "advertisedListeners" feature:
https://pulsar.apache.org/docs/3.1.x/concepts-multiple-advertised-listeners/#advertised-listeners
However, configuring that would require some special customization and integration to make it work with a Pulsar k8s deployment.
Another possibility is the SNI proxy feature and use a proxy that supports SNI proxying (for example Apache Traffic server or Nginx):
https://pulsar.apache.org/docs/3.1.x/concepts-proxy-sni-routing/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1900181490
One problem with Pulsar Proxy is that it adds multiple cross AZ hops which incur network transfer costs in cloud k8s environments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "youzipi (via GitHub)" <gi...@apache.org>.
youzipi commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-2378966965
> Hi @youzipi can you share your manifest for your special ingress created for broker? I need exactly that I think. Thank you
actually, i use proxy now.
this is the ingress config i used that time.
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
labels:
app: pulsar
cluster: pulsar
environment: alpha
component: broker
annotations:
name: "pulsar-broker"
namespace: pulsar
spec:
rules:
- http:
paths:
- path: /pulsar($|/)(.*)
pathType: ImplementationSpecific
backend:
service:
name: "pulsar-broker"
port:
number: 6650
- path: /pulsar-web($|/)(.*)
pathType: ImplementationSpecific
backend:
service:
name: "pulsar-broker"
port:
number: 8080
host: $your-host-name
---
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-2808668957
#601 contains a discussion about how to expose a Pulsar cluster using k8s nodeports. Summary: https://github.com/apache/pulsar-helm-chart/issues/601#issuecomment-2805263229
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "ericsyh (via GitHub)" <gi...@apache.org>.
ericsyh commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-2740547343
In k8s, expose Broker endpoint without using the Pulsar Proxy needs to refer the https://pulsar.apache.org/docs/4.0.x/concepts-proxy-sni-routing/ and a practise for us to use the Istio. Istio ingressgateway will be the external endpoint for clients and istio gateway will do the SNI routing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1900194303
Adding some more context here about the Pulsar Proxy.
https://pulsar.apache.org/docs/3.1.x/administration-proxy/
"Pulsar proxy is used when direct connections between clients and Pulsar brokers are either infeasible or undesirable"
For the "undesirable" part:
At least in the past, some companies have had network security policies which emphasize network perimeter security with
reference architectures where there must be a minimal proxy component for inbound network traffic that has minimal access to any other components and it is placed in a DMZ between 2 firewalls. Many companies still have such security policies in place.
When the Apache Pulsar PMC was handling the Pulsar Proxy security vulnerability https://pulsar.apache.org/security/CVE-2022-24280/, it was decided to add a notice to https://pulsar.apache.org/docs/3.1.x/administration-proxy/ that the Pulsar Proxy isn't designed to be exposed directly on the public internet:
"The Pulsar proxy is not intended to be exposed on the public internet. The security considerations in the current design expect network perimeter security. The requirement of network perimeter security can be achieved with private networks."
For the "infeasible" part:
This is probably about laziness. When something works, many don't care to optimize or improve the solution.
The Pulsar Proxy is very easy to deploy in k8s as we can see in the Apache Pulsar Helm Chart.
The direct connection to brokers could be achieved with advertisedListeners and nodeports. It would be great to have a solution where this could be automated. The nodeport solution would require that the node has a routable address from clients. Since individual brokers don't require stable names, it would be sufficient to be able to advertise the node IP and nodeport.
Lookups could use the REST API configured with an ingress. There is also the possibility to have a loadbalancer for brokers that is used for lookups since that would be more efficient.
Another reason for a proxy like component is for lookups and federating multiple broker clusters into a single large cluster from the client perspective. In Pulsar, there was a component called "pulsar-discovery". This was removed by https://github.com/apache/pulsar/pull/12119 and there's discussion in https://github.com/apache/pulsar/issues/15225 about restoring it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "meyerbro (via GitHub)" <gi...@apache.org>.
meyerbro commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-2376460360
Hi @youzipi can you share your manifest for your special ingress created for broker? I need exactly that I think. Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-6024177979
tracked by #725
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-1900201518
Slightly related: The issue #437 describes a current problem with the headless broker service that should be addressed by adding a 2nd cluster ip service for lookups and making the headless broker service use `publishNotReadyAddresses: true`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] how to direct connect to broker without proxy [pulsar-helm-chart]
Posted by "youzipi (via GitHub)" <gi...@apache.org>.
youzipi commented on issue #423:
URL: https://github.com/apache/pulsar-helm-chart/issues/423#issuecomment-2401412985
> Thanks @youzipi! How are you using the proxy now? Did you find a way to proxy everything through https and not binary tcp? Is it using SNI?
custom configs:
- service for proxy
- ingress for broker
tcp: `host_name:6651` -> alb -> service(proxy, type=LoadBalancer)
http: `host_name:8080/broker-admin` -> ingress(broker) -> service(broker, type=ClusterIP, generated by helm)
```yaml
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alicloud-loadbalancer-address-type: intranet
service.beta.kubernetes.io/alicloud-loadbalancer-id: lb-aaa
service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: "true"
service.beta.kubernetes.io/alicloud-loadbalancer-protocol-port: tcp:6651
service.beta.kubernetes.io/alicloud-loadbalancer-network-type: vpc
name: proxy-tcp
namespace: pulsar-new
labels:
app: pulsar
component: proxy
spec:
type: LoadBalancer
ports:
- name: tcp
targetPort: 6651
port: 6651
protocol: TCP
selector:
app: pulsar
component: proxy
...
```
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
labels:
app: pulsar
cluster: pulsar
environment: prod
component: broker
annotations:
# nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/broker-admin/(.*)$ /$1 break;
name: "pulsar-broker"
namespace: pulsar-new
spec:
rules:
- http:
paths:
- path: /broker-admin
pathType: Prefix
backend:
service:
name: "pulsar-broker"
port:
number: 8080
host: $host-name
---
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org