You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/09/01 19:40:11 UTC
[GitHub] [pulsar] harissecic opened a new issue #7952: OAuth2 url .well-known not parsed correctly
harissecic opened a new issue #7952:
URL: https://github.com/apache/pulsar/issues/7952
When trying to make Pulsar work with Keycloak (supporting multiple realms) I get this:
```bash
# bin/pulsar-admin --admin-url https://localhost:8080 --auth-plugin org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2 --auth-params '{"privateKey":"file:///pulsar_test.json","issuerUrl": "https://*****/auth/realms/***/", "audience":"localhost"}' tenants list
#19:29:25.154 [main] ERROR org.apache.pulsar.client.impl.auth.oauth2.FlowBase - Unable to retrieve OAuth 2.0 server metadata
java.io.IOException: Cannot obtain authorization metadata from https://***/.well-known/openid-configuration
at org.apache.pulsar.client.impl.auth.oauth2.protocol.DefaultMetadataResolver.resolve(DefaultMetadataResolver.java:78) ~[org.apache.pulsar-pulsar-client-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.client.impl.auth.oauth2.FlowBase.initialize(FlowBase.java:50) ~[org.apache.pulsar-pulsar-client-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.client.impl.auth.oauth2.ClientCredentialsFlow.initialize(ClientCredentialsFlow.java:66) ~[org.apache.pulsar-pulsar-client-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2.start(AuthenticationOAuth2.java:96) ~[org.apache.pulsar-pulsar-client-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.client.admin.PulsarAdmin.<init>(PulsarAdmin.java:159) ~[org.apache.pulsar-pulsar-client-admin-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.client.admin.internal.PulsarAdminBuilderImpl.build(PulsarAdminBuilderImpl.java:45) ~[org.apache.pulsar-pulsar-client-admin-original-2.6.1.jar:2.6.1]
at org.apache.pulsar.admin.cli.PulsarAdminTool.lambda$main$2(PulsarAdminTool.java:273) ~[org.apache.pulsar-pulsar-client-tools-2.6.1.jar:2.6.1]
at org.apache.pulsar.admin.cli.PulsarAdminTool.setupCommands(PulsarAdminTool.java:146) [org.apache.pulsar-pulsar-client-tools-2.6.1.jar:2.6.1]
at org.apache.pulsar.admin.cli.PulsarAdminTool.run(PulsarAdminTool.java:226) [org.apache.pulsar-pulsar-client-tools-2.6.1.jar:2.6.1]
at org.apache.pulsar.admin.cli.PulsarAdminTool.main(PulsarAdminTool.java:282) [org.apache.pulsar-pulsar-client-tools-2.6.1.jar:2.6.1]
Caused by: java.io.FileNotFoundException: https://zdb-users.azurewebsites.net/.well-known/openid-configuration
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1896) ~[?:1.8.0_252]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_252]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268) ~[?:1.8.0_252]
at org.apache.pulsar.client.impl.auth.oauth2.protocol.DefaultMetadataResolver.resolve(DefaultMetadataResolver.java:72) ~[org.apache.pulsar-pulsar-client-original-2.6.1.jar:2.6.1]
... 9 more
class org.apache.pulsar.client.api.PulsarClientException$AuthenticationException: Unable to retrieve OAuth 2.0 server metadata
```
So it appears that extra path after the base URL is ignored and stripped. It would be really useful to have it respect full path and work with Keycloak. As visible, /auth/realms/{RELAM_NAME} is ignored and it's trying to access base URL by appending /.well-known/openid-configuration instead of appending it at the end of provided issuer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cuzyoucant commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
cuzyoucant commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686508429
@zymap I can agree what @harissecic says. The problem is the line 100 in the DefaultMetadataResolver.java
`return new URL(issuerUrl, "/.well-known/openid-configuration");`
Because the URL constructor does not work as someone would expect. It does not concatenate the two inputs. You can read that up in the java documentation. It strips everything from the issuerURL after the `www.xyz.com` and adds the `"/.well-known/openid-configuration"`.
So no matter what you pass in as issuerURL` www.xyz.com` or` www.xyz.com/auth/realms/myrealm/` (which we want to pass in, or rather what you need to pass in if you use keycloak) it always ends up as `www.xyz.com/.well-known/openid-configuration`.
Thats why i just changed it the line to `return new URL(issuerUrl.toString() + "/.well-known/openid-configuration");` so it actually gets appended.
Hope that helps!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic edited a comment on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
harissecic edited a comment on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686447346
> Hi, @harissecic By this error `java.io.IOException: Cannot obtain authorization metadata from https://***/.well-known/openid-configuration`, it seems the client can not access the configuration URL. Have you tried to using curl to get it directly?
I'm afraid you didn't read it entirely.
1. https://***/.well-known/openid-configuration is incorrect and that's on Pulsar side 100%
2. https://*****/auth/realms/*** is passed in and is quite clear and pointed out that /auth/realms/*realmId* is being stripped inside Pulsar code and /.well-known/openid-configuration is being appended at the base URL when my realms are stripped as I stated in the issue as first sentence after the log.
> So it appears that extra path after the base URL is ignored and stripped
3. Endpoint I provide is 100% correct and working. My APIs are using that same URL.
Please note that stars are used to hide the actual BASE URL and REALM part is left to be pointed out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cuzyoucant commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
cuzyoucant commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-685904145
Hey.
We ran into the same problem as you described above. The better solution would be to have a config somewhere where we can set what gets appended to the issuerURL.
I implemented a small ghetto fix for myself and build pulsar from source until this gets officially implemented.
If this is something you want to do, here is what you need to change:
To "fix" your problem you need to change the line 100 in the DefaultMetadataResolver.java
from this: `return new URL(issuerUrl, "/.well-known/openid-configuration");`
to this: `return new URL(issuerUrl.toString() + "/.well-known/openid-configuration");`
Now the URL gets appended correctly and it works as expected.
To also get the client(producer/consumer) to work you need to change the following since they also do not work with keycloak URLs:
1. In the file /pulsar-client-cpp/lib/auth/AuthOauth2.cc in line 187 comment out the line.
`issuerUrl_.append("/oauth/token");`
2. Now you need to specify the full token endpoint from keycloak as the issuer_url in your client.
example: `https://example.com/auth/realms/myrealm/protocol/openid-connect/token`
The reason you need to do that is that after the first token expires, it tries to get a new one with the settings from the /.well-known configuration and it appends /oauth/token again to the tokenURL which obviously doesnt work.
I know its kind of a bad fix for now but i am to inexperienced to add a complete feature myself. We just want it to work for now and we will not be using it for production anytime soon. Just want to test features and infrastructure stuff and for that it works fine.
Hope it helps!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
zymap commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686361630
Hi, @harissecic By this error `java.io.IOException: Cannot obtain authorization metadata from https://***/.well-known/openid-configuration`, it seems the client can not access the configuration URL. Have you tried to using curl to get it directly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie closed issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
sijie closed issue #7952:
URL: https://github.com/apache/pulsar/issues/7952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie closed issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
sijie closed issue #7952:
URL: https://github.com/apache/pulsar/issues/7952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
zymap commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686562632
@harissecic @cuzyoucant Sorry for my mistake and thanks for your patience.
Would you like to create a patch for the Java implementation?
For the CPP client, it will be better if we have the same implementation as Java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
harissecic commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686447346
> Hi, @harissecic By this error `java.io.IOException: Cannot obtain authorization metadata from https://***/.well-known/openid-configuration`, it seems the client can not access the configuration URL. Have you tried to using curl to get it directly?
I'm afraid you didn't read it entirely.
1. https://***/.well-known/openid-configuration is incorrect and that's on Pulsar side 100%
2. https://*****/auth/realms/***/ is passed in and is quite clear and pointed out that /auth/realms/*realmId* is being stripped inside Pulsar code and /.well-known/openid-configuration is being appended at the base URL when my realms are stripped as I stated in the issue as first sentence after the log.
> So it appears that extra path after the base URL is ignored and stripped
3. Endpoint I provide i 100% correct and working. My APIs are using that same URL.
Please not that stars are used to hide the actual BASE URL and REALM part is left to be pointed out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686271862
@jiazhai Can you take a look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie closed issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
sijie closed issue #7952:
URL: https://github.com/apache/pulsar/issues/7952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic edited a comment on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
harissecic edited a comment on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686447346
> Hi, @harissecic By this error `java.io.IOException: Cannot obtain authorization metadata from https://***/.well-known/openid-configuration`, it seems the client can not access the configuration URL. Have you tried to using curl to get it directly?
I'm afraid you didn't read it entirely.
1. https://***/.well-known/openid-configuration is incorrect and that's on Pulsar side 100%
2. https://*****/auth/realms/*** is passed in and is quite clear and pointed out that /auth/realms/*realmId* is being stripped inside Pulsar code and /.well-known/openid-configuration is being appended at the base URL when my realms are stripped as I stated in the issue as first sentence after the log.
> So it appears that extra path after the base URL is ignored and stripped
3. Endpoint I provide i 100% correct and working. My APIs are using that same URL.
Please not that stars are used to hide the actual BASE URL and REALM part is left to be pointed out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
harissecic commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-685947635
@cuzyoucant Thank you for detailed description. I'll consider it. However I'm currently using JWT until this gets fixed and I'm trying to make integration with a certain framework which is better to put off than to test with custom code. First case for my client might benefit from building Pulsar from code. Thanks again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] EronWright commented on issue #7952: OAuth2 url .well-known not parsed correctly
Posted by GitBox <gi...@apache.org>.
EronWright commented on issue #7952:
URL: https://github.com/apache/pulsar/issues/7952#issuecomment-686827234
Regarding the openid configuration document location, @cuzyoucant is correct that the base path of the `issuerUrl` should be preserved, to better interoperate with Keycloak and others. The following expression would work robustly:
`URI.create(issuerUrl.toExternalForm() + "/.well-known/openid-configuration").normalize().toURL();`
Regarding the CPP client issue, I believe the token endpoint should be obtained from the openid configuration document, specifically the `token_endpoint` value, as documented here:
https://tools.ietf.org/html/rfc8414#section-2
The overarching idea is, from the configured `issuerUrl` we get the openid configuration document, and from that we get the token endpoint.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org