You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/29 13:39:43 UTC
[GitHub] [pulsar] frankjkelly opened a new issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
frankjkelly opened a new issue #7385:
URL: https://github.com/apache/pulsar/issues/7385
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. GIVEN we enable Authorization on proxy
```
root@pulsar-proxy-6f798754db-r9gbw:/pulsar/conf# grep -i authorization proxy.conf
### ---Authorization --- ###
# Whether authorization is enforced by the Pulsar proxy
authorizationEnabled=true
# Authorization provider as a fully qualified class name
authorizationProvider=com.cogito.platform.signal.stream.pulsar.authn.broker.CogitoAuthorizationProvider
# Whether client authorization credentials are forwared to the broker for re-authorization.
forwardAuthorizationCredentials=false
```
2. AND GIVEN the Authorization plugin is loaded
```
[16:02:04] fkelly@Franks-Cogito-Work-Computer:[~/platform2-test]: (feature/sdlc-31257-minikube-integration) klf pulsar-proxy-6f798754db-r9gbw | grep -i authorization
[conf/proxy.conf] Applying config authorizationEnabled = true
[conf/proxy.conf] Applying config authorizationProvider = com.cogito.platform.signal.stream.pulsar.authn.broker.CogitoAuthorizationProvider
19:55:31.069 [main] INFO com.cogito.platform.signal.stream.pulsar.authn.broker.CogitoAuthorizationProvider - ==> Initialize()
19:55:31.074 [main] INFO org.apache.pulsar.broker.authorization.AuthorizationService - com.cogito.platform.signal.stream.pulsar.authn.broker.CogitoAuthorizationProvider has been loaded.
```
3. WHEN a Pulsar command is picked up by the Proxy
4. THEN the Authentication Plugin is executed but the Authorization Plugin is not (and the Role token is passed onto the Broker)
**Expected behavior**
Unless I am misunderstanding the Auth*n deployment / configuration then I would expect Authorization to occur at the Proxy.
**Screenshots**
See above
**Desktop (please complete the following information):**
- Apache Pulsar: 2.5.2
- Kubernetes: minikube version: v1.9.0
**Additional context**
Add any other context about the problem here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676650448
Thanks @sijie yes I think you are right - is that changed scheduled to go out with 2.6.1?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-652835256
If you set `forwardAuthorizationCredentials` to `true`, proxy will forward the authentication credentials to the brokers and brokers will do both authentication and authorization. But in proxy, you need to do configure a super-user token, because proxy is required to read the partitioned topic metadata and lookup topic ownerships.
So you need to configure the following:
```
forwardAuthorizationCredentials=true
brokerClientAuthenticationPlugin=[plugin]
brokerClientAuthenticationParameters=[parameters]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676616269
Could be a dupe of https://github.com/apache/pulsar/issues/7830
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly removed a comment on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly removed a comment on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676616269
Could be a dupe of https://github.com/apache/pulsar/issues/7830
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly closed issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly closed issue #7385:
URL: https://github.com/apache/pulsar/issues/7385
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-651465495
@frankjkelly Are you using zookeeper or brokerServiceURL for broker discovery? If you don't set zookeeper, proxy will not enable authorization. Because the default authorization plugin is implemented based on zookeeper.
There might be places that we can improve. E.g. we don't need zookeeper if users provide a customized implementation of Authorization plugin.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly edited a comment on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly edited a comment on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-651799700
Hmmm I'm not sure. In proxy.conf we have the following for `brokerService`
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep broker proxy.conf | grep -v "#"
brokerServiceURL=http://pulsar-broker:6650
brokerServiceURLTLS=https://pulsar-broker:6651
brokerWebServiceURL=http://pulsar-broker:8080
brokerWebServiceURLTLS=https://pulsar-broker:8443
authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
and the following zookeeper related
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep zoo proxy.conf
zookeeperServers=pulsar-zookeeper-0.pulsar-zookeeper
configurationStoreServers=pulsar-zookeeper-0.pulsar-zookeeper
# These settings are unnecessary if `zookeeperServers` is specified
zookeeperSessionTimeoutMs=30000
```
I see from [this StreamNative documentation](https://streamnative.io/docs/v1.0.0/configure/pulsar-core/proxy/) that by defining the `brokerServiceURL` we are effectively disabling Authorization. I guess we can defer the Authorization to the Broker but the problem I believe I have is that our JWT token contains custom claims that indicate whether a topic is accessible or not (it's not Role-based) so we need to pass the token along to the Authorization plugin for parsing. In standalone - when the Authentication and Authorization plugin are colocated this is fine but it seems that when the Proxy is doing Authentication and the Broker is doing Authorization the original token is no longer present at the Broker? I tried `forwardAuthorizationCredentials=true` but it did not seem to make a difference but perhaps I am missing something.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676626377
If you are using 2.6.0, I think there are some issues with handling proxy forwarded credentials. The change in #7788 fixes it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-651799700
Hmmm I'm not sure. In proxy.conf we have the following for `brokerService`
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep broker proxy.conf | grep -v "#"
brokerServiceURL=http://pulsar-broker:6650
brokerServiceURLTLS=https://pulsar-broker:6651
brokerWebServiceURL=http://pulsar-broker:8080
brokerWebServiceURLTLS=https://pulsar-broker:8443
authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
and the following zookeeper related
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep zoo proxy.conf
zookeeperServers=pulsar-zookeeper-0.pulsar-zookeeper
configurationStoreServers=pulsar-zookeeper-0.pulsar-zookeeper
# These settings are unnecessary if `zookeeperServers` is specified
zookeeperSessionTimeoutMs=30000
```
I see from this StreamNative documentation that by defining the `brokerServiceURL` we are effectively disabling Authorization.
The problem I have is that our JWT token contains custom claims that indicate whether a topic is accessible or not (it's not Role-based) so we need to pass the token along to the Authorization plugin for parsing. In standalone - when the Authentication and Authorization plugin are colocated this is fine but it seems that when the Proxy is doing Authentication and the Broker is doing Authorization the original token is no longer present at the Broker? I tried `forwardAuthorizationCredentials=true` but it did not seem to make a difference but perhaps I am missing something.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-655647766
@vzhikserg: the `proxyRole` is used for topic lookup (to figure who is the owner broker for a given topic) before establishing the proxied connections to the brokers. Hence the `proxyRole` is required to perform topic lookup over all topics. That means the `proxyRole` has to be a super-user role. Does that make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676616769
Could be related to https://github.com/apache/pulsar/issues/7830
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly edited a comment on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly edited a comment on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-651799700
Hmmm I'm not sure. In proxy.conf we have the following for `brokerService`
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep broker proxy.conf | grep -v "#"
brokerServiceURL=http://pulsar-broker:6650
brokerServiceURLTLS=https://pulsar-broker:6651
brokerWebServiceURL=http://pulsar-broker:8080
brokerWebServiceURLTLS=https://pulsar-broker:8443
authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
and the following zookeeper related
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep zoo proxy.conf
zookeeperServers=pulsar-zookeeper-0.pulsar-zookeeper
configurationStoreServers=pulsar-zookeeper-0.pulsar-zookeeper
# These settings are unnecessary if `zookeeperServers` is specified
zookeeperSessionTimeoutMs=30000
```
I see from [this StreamNative documentation](https://streamnative.io/docs/v1.0.0/configure/pulsar-core/proxy/) that by defining the `brokerServiceURL` we are effectively disabling Authorization in the proxy?
If that's right I guess we can defer the Authorization to the Broker but the problem I believe I have is that our JWT token contains custom claims that indicate whether a topic is accessible or not (it's not Role-based) so we need to pass the token along to the Authorization plugin for parsing. In standalone - when the Authentication and Authorization plugin are colocated this is fine but it seems that when the Proxy is doing Authentication and the Broker is doing Authorization the original token is no longer present at the Broker? I tried `forwardAuthorizationCredentials=true` but it did not seem to make a difference but perhaps I am missing something.
Thanks in advance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676699148
Yes. It is part of the 2.6.1 release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-676699695
Wonderful thanks I'm super excited to test it
CC: @Joshhw
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vzhikserg commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
vzhikserg commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-655521061
We see the same behavior using 2.6.0 on AKS deployment when connecting (jwt) via proxy with authentication and authorization enabled (using zookeeper):
- Client has no permissions an a topic the client is rejected, as expected
- Client has been granted for example only produce, then the client is able to consume as well (besides producing) I assume because the superUserRole is being used.
When we configure the proxyRoles and apply the same permissions as the client it works, however this implies that proxy clients get the proxyRoles or superUserRoles.
Meaning that in case we have 2 clients connecting to the same topic: one with consume and the other with produce permissions, then proxyRole would need produce and consume permissions, which leads to that both clients can produce and consume.
P.S. we watched the video twice ;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-680892498
Closing this issue - more because my initial understanding / statement of the problem was incorrect
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly edited a comment on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
frankjkelly edited a comment on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-651799700
Hmmm I'm not sure. In proxy.conf we have the following for `brokerService`
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep broker proxy.conf | grep -v "#"
brokerServiceURL=http://pulsar-broker:6650
brokerServiceURLTLS=https://pulsar-broker:6651
brokerWebServiceURL=http://pulsar-broker:8080
brokerWebServiceURLTLS=https://pulsar-broker:8443
authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
and the following zookeeper related
```
root@pulsar-proxy-764f4b6569-flg8m:/pulsar/conf# grep zoo proxy.conf
zookeeperServers=pulsar-zookeeper-0.pulsar-zookeeper
configurationStoreServers=pulsar-zookeeper-0.pulsar-zookeeper
# These settings are unnecessary if `zookeeperServers` is specified
zookeeperSessionTimeoutMs=30000
```
I see from [this StreamNative documentation](https://streamnative.io/docs/v1.0.0/configure/pulsar-core/proxy/) that by defining the `brokerServiceURL` we are effectively disabling Authorization. I guess we can defer the Authorization to the Broker but the problem I believe I have is that our JWT token contains custom claims that indicate whether a topic is accessible or not (it's not Role-based) so we need to pass the token along to the Authorization plugin for parsing. In standalone - when the Authentication and Authorization plugin are colocated this is fine but it seems that when the Proxy is doing Authentication and the Broker is doing Authorization the original token is no longer present at the Broker? I tried `forwardAuthorizationCredentials=true` but it did not seem to make a difference but perhaps I am missing something.
Thanks in advance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7385: When `authorizationEnabled=true` in proxy.conf the proxy does not appear to perform Authorization check
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7385:
URL: https://github.com/apache/pulsar/issues/7385#issuecomment-652835665
You can also check this video: https://www.youtube.com/watch?v=sTISVpyq73o
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org