You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "MMirelli (via GitHub)" <gi...@apache.org> on 2024/05/09 11:30:24 UTC
[I] Allow broker's service clusterIP customisation: headless vs non-headless [pulsar-helm-chart]
MMirelli opened a new issue, #497:
URL: https://github.com/apache/pulsar-helm-chart/issues/497
**Is your feature request related to a problem? Please describe.**
When deploying pulsar on a local k8s cluster and exposing the pulsar proxy service port via a node port (https://k3d.io/v5.0.1/usage/exposing_services/#2-via-nodeport), a headless ClusterIP broker service doesn't seem to be enough. In other words, the proxy doesn't manage to resolve the broker's IP (most probably because that is effectively not being assigned by kubernetes).
Hitting the pulsar-proxy exposed locally results in these error logs on the proxy:
```
2024-05-08T21:39:46,190+0000 [pulsar-proxy-io-2-1] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/10.42.2.0:40935] Connection closed
2024-05-08T21:39:46,380+0000 [pulsar-proxy-io-2-2] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/10.42.0.1:44829] Received cmd LOOKUP
2024-05-08T21:39:46,381+0000 [pulsar-proxy-io-2-2] DEBUG org.apache.pulsar.proxy.server.LookupProxyHandler - Received Lookup from /10.42.0.1:44829
2024-05-08T21:39:46,381+0000 [pulsar-proxy-io-2-2] DEBUG org.apache.pulsar.proxy.server.LookupProxyHandler - Getting connections to 'pulsar-test-broker/<unresolved>:6650' for Looking up topic 'persistent://public/default/myt' with clientReq Id '3188434002705864041'
2024-05-08T21:39:46,382+0000 [pulsar-proxy-io-2-5] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [pulsar-test-broker.pulsar.svc.cluster.local/10.42.2.8:6650] Received cmd LOOKUP_RESPONSE
2024-05-08T21:39:46,382+0000 [pulsar-proxy-io-2-5] DEBUG org.apache.pulsar.client.impl.ClientCnx - Received Broker lookup response: 3773831502744970664 Connect
2024-05-08T21:39:46,382+0000 [pulsar-proxy-io-2-5] DEBUG org.apache.pulsar.proxy.server.LookupProxyHandler - Successfully perform lookup 'pulsar-test-broker/<unresolved>:6650' for topic 'persistent://public/default/myt' with clientReq Id '3188434002705864041' and lookup-broker pulsar://0.0.0.0:6650
2024-05-08T21:39:46,384+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.common.protocol.PulsarHandler - [[id: 0x786f9063, L:/10.42.0.9:6650 - R:/10.42.0.1:13386]] Scheduling keep-alive task every 30 s
2024-05-08T21:39:46,384+0000 [pulsar-proxy-io-2-3] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/10.42.0.1:13386] New connection opened
2024-05-08T21:39:46,384+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/10.42.0.1:13386] Received cmd CONNECT
2024-05-08T21:39:46,384+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.proxy.server.ProxyConnection - Received CONNECT from /10.42.0.1:13386 proxyToBroker=0.0.0.0:6650
2024-05-08T21:39:46,384+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.proxy.server.ProxyConnection - [/10.42.0.1:13386] Protocol version to advertise to broker is 21, clientProtocolVersion=21, proxyProtocolVersion=21
2024-05-08T21:39:46,385+0000 [pulsar-proxy-io-2-3] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/10.42.0.1:13386] complete connection, init proxy handler. authenticated with none role null, hasProxyToBrokerUrl: true
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] DEBUG org.apache.pulsar.common.protocol.PulsarHandler - [[id: 0xf6ff2c17, L:/127.0.0.1:6650 - R:/127.0.0.1:36858]] Scheduling keep-alive task every 30 s
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:36858] New connection opened
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/127.0.0.1:36858] Received cmd CONNECT
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] DEBUG org.apache.pulsar.proxy.server.ProxyConnection - Received CONNECT from /127.0.0.1:36858 proxyToBroker=null
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] DEBUG org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:36858] Protocol version to advertise to broker is 21, clientProtocolVersion=21, proxyProtocolVersion=21
2024-05-08T21:39:46,386+0000 [pulsar-proxy-io-2-4] DEBUG org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:36858] Client illegally provided proxyVersion.
2024-05-08T21:39:46,387+0000 [pulsar-proxy-io-2-4] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:36858] Connection closed
2024-05-08T21:39:46,387+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.proxy.server.DirectProxyHandler - [[id: 0x786f9063, L:/10.42.0.9:6650 - R:/10.42.0.1:13386]] [[id: 0xdc2a5ffc, L:/127.0.0.1:36858 - R:0.0.0.0/127.0.0.1:6650]] Received msg on broker connection: class io.netty.buffer.PooledSlicedByteBuf
2024-05-08T21:39:46,387+0000 [pulsar-proxy-io-2-3] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [0.0.0.0/127.0.0.1:6650] Received cmd ERROR
2024-05-08T21:39:46,387+0000 [pulsar-proxy-io-2-3] WARN org.apache.pulsar.proxy.server.DirectProxyHandler - [[id: 0x786f9063, L:/10.42.0.9:6650 - R:/10.42.0.1:13386]] [[id: 0xdc2a5ffc, L:/127.0.0.1:36858 - R:0.0.0.0/127.0.0.1:6650]] Caught exception: null
java.lang.UnsupportedOperationException: null
at org.apache.pulsar.common.protocol.PulsarDecoder.handleError(PulsarDecoder.java:592) ~[org.apache.pulsar-pulsar-common-3.0.3.jar:3.0.3]
at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:192) ~[org.apache.pulsar-pulsar-common-3.0.3.jar:3.0.3]
at org.apache.pulsar.proxy.server.DirectProxyHandler$ProxyBackendHandler.channelRead(DirectProxyHandler.java:354) ~[org.apache.pulsar-pulsar-proxy-3.0.3.jar:3.0.3]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[io.netty-netty-codec-4.1.100.Final.jar:4.1.100.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[io.netty-netty-codec-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[io.netty-netty-handler-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) ~[io.netty-netty-handler-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[io.netty-netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[io.netty-netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[io.netty-netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[io.netty-netty-common-4.1.100.Final.jar:4.1.100.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.100.Final.jar:4.1.100.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.100.Final.jar:4.1.100.Final]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
```
Also, a headless service doesn't support configuring the broker's `advertisedAddressed` to match the broker's service name. For example setting `advertisedListeners: "internal:pulsar://<pulsar_broker_service>:6650"` in the broker config, doesn't allow the broker to startup smoothly, as it cannot resolve the service name to an IP address.
> This makes it very hard / counter-intuitive to deploy the chart on a local kubernetes cluster while exposing the pulsar service via the proxy. The current headless solution, works just fine when exposing the pulsar service via broker, instead.
**Describe the solution you'd like**
Allow custom rewriting of the IP assigned to the pulsar-broker ClusterIP service. We could allow 3 modes:
1. a valid IPv4 address
2. `auto` mode, i.e. `clusterIP: ''`
3. headless, i.e. `clusterIP: 'None'`
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
More about the `clusterIP: ''` vs `clusterIP: 'None'` (modes 2. and 3. above) can be found at https://kubernetes.io/docs/concepts/services-networking/service/#type-clusterip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Allow broker's service clusterIP customisation: headless vs non-headless [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #497:
URL: https://github.com/apache/pulsar-helm-chart/issues/497#issuecomment-2118784954
When using a node port, the type of the service would also have to be NodePort ([docs](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types)).
This means that there might have to be multiple services in the case of needing a separate service for internal access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Allow broker's service clusterIP customisation: headless vs non-headless [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari closed issue #497: Allow broker's service clusterIP customisation: headless vs non-headless
URL: https://github.com/apache/pulsar-helm-chart/issues/497
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org