You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/08 08:16:11 UTC
[GitHub] [pulsar] isminexxb opened a new issue, #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
isminexxb opened a new issue, #17540:
URL: https://github.com/apache/pulsar/issues/17540
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
server: v2.10.0
client: c++
### Minimal reproduce step
1. We already have an internal cluster without any auth mechanism.
2. Now we need to open the cluster to external services. Considering that existing internal clients connect to the cluster without any verification, we try to add proxy service for external verification. We configure the proxy with zk url instead of broker url.
3. At first, we follow the guide (https://pulsar.apache.org/docs/next/security-tls-transport) to configure the proxy for TLS encryption, it works as expected.
4. Then, we follow the guide (https://pulsar.apache.org/docs/next/security-jwt) to configure the proxy for JWT, the proxy throws exception as follows and close the client connection.
2022-09-08T15:42:39,307+0800 [pulsar-proxy-io-2-11] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:48159] New connection opened
2022-09-08T15:42:39,311+0800 [pulsar-proxy-io-2-11] WARN io.netty.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:480) ~[io.netty-netty-codec-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) ~[io.netty-netty-codec-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [io.netty-netty-transport-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) [io.netty-netty-transport-classes-epoll-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) [io.netty-netty-transport-classes-epoll-4.1.74.Final.jar:4.1.74.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) [io.netty-netty-transport-classes-epoll-4.1.74.Final.jar:4.1.74.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.74.Final.jar:4.1.74.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.needWrapAgain(ReferenceCountedOpenSslEngine.java:1343) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1360) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1305) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1392) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1435) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:221) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1342) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1246) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286) ~[io.netty-netty-handler-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) ~[io.netty-netty-codec-4.1.74.Final.jar:4.1.74.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) ~[io.netty-netty-codec-4.1.74.Final.jar:4.1.74.Final]
... 15 more
2022-09-08T15:42:39,311+0800 [pulsar-proxy-io-2-11] WARN org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:48159] Got exception DecoderException : javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE
2022-09-08T15:42:39,311+0800 [pulsar-proxy-io-2-11] INFO org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:48159] Connection closed
5. After analysis, we realize that both TLS encryption and JWT require the client to use "setAuth()" method to set authentication method. Does it mean that JWT cannot work on top of TLS encryption?
6. At last, We abandon the TLS encryption and test with JWT only. JWT authentication works as expected.
7. But JWT authorization does not work. Client can produce message to the topic even it is not granted the privilege. We review the implemention of proxy service(/pulsar/pulsar-proxy), no authorization check was found.
### What did you expect to see?
1. JWT works well with TLS encryption.
2. JWT authorization works well with proxy service only.
### What did you see instead?
Exception occurres or JWT authorization does not work.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb closed issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb closed issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
URL: https://github.com/apache/pulsar/issues/17540
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1240646189
> Only Pulsar 2.11 supports JWT with TLS transport, you need to use the TLS authentication now.
Did you mean [#15289](https://github.com/apache/pulsar/issues/15289)? Does it still not support the proxy and c++ library? In addition, will JWT authorization be supported for proxy then? @nodece
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1240533086
Only Pulsar 2.11 supports JWT with TLS transport, you need to use the TLS authentication now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241437645
> Is this usage not supported now?
We support this feature, you need to add `forwardAuthorizationCredentials=true` to the `proxy.conf` file, and add the `authenticateOriginalAuthData=true` to the `broker.conf`.
I'll improve this doc today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241884592
> The proxy only does authentication, and cannot do authorization by itself.
That is the key point. I thought the proxy can do authorization by itself, so I tried to let the proxy undertake encryption, authentication and authorization alone.
By the way, why cannot the proxy do authorization by itself? What are the considerations?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1242149728
> By the way, why cannot the proxy do authorization by itself? What are the considerations?
Because the client and the broker communicate through the proxy, and the proxy is a transparent component, this means that the client communicates directly with the broker.
So the proxy needs forward the authentication data from the client to the broker, and the broker handles the proxy and the client authorization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1240883273
> > Only Pulsar 2.11 supports JWT with TLS transport, you need to use the TLS authentication now.
>
> Did you mean [#15289](https://github.com/apache/pulsar/issues/15289)? Does it still not support the proxy and c++ library? In addition, will JWT authorization be supported for proxy then? @nodece
I improve the broker and proxy, but the CPP client misses this improvement, I can make a PR to improve that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241653652
> > Is this usage not supported now?
>
> We support this feature, you need to add `forwardAuthorizationCredentials=true` to the `proxy.conf` file, and add the `authenticateOriginalAuthData=true` to the `broker.conf`.
>
> I'll improve this doc today.
Does it seems that I need to enable authorization at the broker level? Can authorization checked by proxy itself? We already have many clients which connect to broker directly and we'd better not change them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241731090
> Does it seems that I need to enable authorization at the broker level?
Yes, you need to enable authorization on the broker.
> Can authorization checked by proxy itself?
No, you have to enable authorization on the broker.
Do you need the authorization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241749593
> > Does it seems that I need to enable authorization at the broker level?
>
> Yes, you need to enable authorization on the broker.
>
> > Can authorization checked by proxy itself?
>
> No, you have to enable authorization on the broker.
>
> Do you need the authorization?
Yes for new clients of public network. And I don't want to affect previous intranet clients connected to brokers directly without authorization. I tried to use proxy for new clients of public network to realize this requirement before, but it seems not feasible at present.
Look forward to your suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241794984
Let's back to the original issue.
> JWT works well with TLS encryption.
Is public and trusted for the TLS certificates? Do you need the mTLS?
> JWT authorization works well with proxy service only.
should be JWT authentication, not authorization?
The proxy only does authentication, and cannot do authorization by itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] isminexxb commented on issue #17540: [Bug] It doesn't work when using JWT auth on top of TLS transport encryption with proxy service
Posted by GitBox <gi...@apache.org>.
isminexxb commented on issue #17540:
URL: https://github.com/apache/pulsar/issues/17540#issuecomment-1241406187
> > > Only Pulsar 2.11 supports JWT with TLS transport, you need to use the TLS authentication now.
> >
> >
> > Did you mean [#15289](https://github.com/apache/pulsar/issues/15289)? Does it still not support the proxy and c++ library? In addition, will JWT authorization be supported for proxy then? @nodece
>
> I improved the broker and proxy, but the CPP client misses this improvement, I can make a PR to improve that.
Nice to hear that.
But when I use JWT with proxy only (Broker works without any authentication or authorization configuration. Proxy configures with authorizationEnabled=true but does not forward authorization credentials to brokers), the client can still produce messages without privilege. I also haven't found related authorization check in proxy's implementation or proxy's log. Is this usage not supported now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org