You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/10/18 08:21:00 UTC
[GitHub] [pulsar-dotpulsar] VisualBean opened a new issue #92: Client usage
VisualBean opened a new issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92
Hi guys, im wondering what the expected usage of this library is;
* How should I consider my client? Singleton/scoped/transient?
* How should I consider the consumer/producer instances?
* The client, doesn't support any delegate for rolling a new jwt, which indicates to me the client is to be considered either scoped or transient - but what are the characteristics of port usage/connection pooling?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-946437489
Yes, the other clients has this support through a plugin called `AuthenticationOauth2`. However, I would be fine with simply having a delegate I could use, like the Microsoft Graph Client for instance;
```csharp
var pca = PublicClientApplicationBuilder
.Create(clientId)
.WithTenantId(tenantId)
.Build();
var authProvider = new DelegateAuthenticationProvider(async (requestMessage) =>
{
var result = await pca.AcquireTokenByUsernamePassword(defaultGraphScopes, username, password).ExecuteAsync();
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("bearer", result.AccessToken);
});
new GraphServiceClient(authProvider);
```
where you basically just pass it the logic for getting a new token. - I would be perfectly fine with managing the caching myself as well (which is also why im thinking to just wrap the client creation in yet another factory)
I guess the connection pool is referenced in the PulsarClientBuilder, right? So I guess as long as we keep this instance around, it is fine.
My worry is creating too many clients, since we have to create a new one when our tokens expire (every 30 minutes or so). 30 minutes is more than enough for resetting the socket, so we dont get port exhaustion, but it made me ask anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1028829037
You are most welcome .:-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950331145
I trying to figure it out with someone from the pulsar community, when exactly this check happens (the protocol docs aren't very clear on this) - but
> After receiving a Connected response from the broker, the client can consider the connection ready to use. Alternatively, if the broker doesn't validate the client authentication, it will reply with an Error command and close the TCP connection. - https://pulsar.apache.org/docs/en/develop-binary-protocol/#connection-establishment
This doesn't seem to be supported in the dotpulsar client, as I simply got a timeout after waiting a long time. - I will test it again however to make sure. - I tested it simply with a wrong token.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-951003820
Followup for the broker side checking expiry.
https://github.com/apache/pulsar/blob/464a9cdc21609189771f1d95ac211b69eca1454c/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java#L368
Which is used here (among other places)
https://github.com/apache/pulsar/blob/464a9cdc21609189771f1d95ac211b69eca1454c/pulsar-broker-auth-sasl/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderSasl.java#L164
(constant here https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-common/src/main/java/org/apache/pulsar/common/sasl/SaslConstants.java#L66)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950636235
It appears there are 2 flows according to one of the original Pulsar creators Matteo Merli.
There is the AuthenticationProvider, which that's sends a request to the client to re-validate the token when it is about to expire.
And if this is not supported by the client or the token is allowed to expire, the broker will close the TCP connection.
For posterity, the Java client looks something like this
```java
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://broker.example.com:6650/")
.authentication(
AuthenticationFactoryOAuth2.clientCredentials(issuerUrl, credentialsUrl, audience))
.build();
```
With the internals found here https://github.com/apache/pulsar/blob/ac5114f8944784972b831438f8c7e0cbd57db4e5/pulsar-client/src/main/java/org/apache/pulsar/client/impl/auth/oauth2/AuthenticationOAuth2.java#L40
For an idiomatic .net experience, I would probably still be more inclined to opt for the delegate.
Is this something you see easily implemented in the current state of the dotpulsar client?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950651849
Hi @VisualBean
Thanks for the research!
I think we need proper support for this, meaning that we support re-validation of the token on the fly (didn't know that was possible). This will secure a constant flow instead of connections closing and states changing.
I hope to get some time to look at this soon, so that it can be part of the next major release happening next month.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-949581001
The default behavior, when a TCP connection is closed, is to go into the 'Disconnected' state and then try to re-establish the connection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950331145
I am trying to figure it out with someone from the pulsar community, when exactly this check happens (the protocol docs aren't very clear on this) - but
> After receiving a Connected response from the broker, the client can consider the connection ready to use. Alternatively, if the broker doesn't validate the client authentication, it will reply with an Error command and close the TCP connection. - https://pulsar.apache.org/docs/en/develop-binary-protocol/#connection-establishment
This doesn't seem to be supported in the dotpulsar client, as I simply got a timeout after waiting a long time. - I will test it again to make sure this is what I am seeing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950636235
It appears there are 2 flows according to one of the original Pulsar creators Matteo Merli.
There is the AuthenticationProvider, which that's sends a request to the client to re-validate the token when it is about to expire.
And if this is not supported by the client, the broker will close the TCP connection when the token expires.
For posterity, the Java client looks something like this
```java
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://broker.example.com:6650/")
.authentication(
AuthenticationFactoryOAuth2.clientCredentials(issuerUrl, credentialsUrl, audience))
.build();
```
With the internals found here https://github.com/apache/pulsar/blob/ac5114f8944784972b831438f8c7e0cbd57db4e5/pulsar-client/src/main/java/org/apache/pulsar/client/impl/auth/oauth2/AuthenticationOAuth2.java#L40
For an idiomatic .net experience, I would probably still be more inclined to opt for the delegate.
Is this something you see easily implemented in the current state of the dotpulsar client?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1028808586
Looks good. From my quick glance, it looks like we can simply implement a different authfactory for oauth/whatever we want?
I see another issue has been created for OAuth specifically #94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1028828300
Awesome, I will most likely contribute within a couple of days then!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950124682
Ok, that sounds a bit strange. I would expect a reply telling us that the token is not accepted.
How about when the token is valid upon connection creation but expires during the lifetime of the connection?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-946437489
Yes, the other clients has this support through a plugin called `AuthenticationOauth2`. However, I would be fine with simply having a delegate I could use, like the Microsoft Graph Client for instance;
```csharp
var pca = PublicClientApplicationBuilder
.Create(clientId)
.WithTenantId(tenantId)
.Build();
var authProvider = new DelegateAuthenticationProvider(async (requestMessage) =>
{
var result = await pca.AcquireTokenByUsernamePassword(defaultGraphScopes, username, password).ExecuteAsync();
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("bearer", result.AccessToken);
});
new GraphServiceClient(authProvider);
```
where you basically just pass it the logic for getting a new token. - I would be perfectly fine with managing the caching myself as well (which is also why im thinking to just wrap the client creation in yet another factory)
The connection pool is referenced in the PulsarClientBuilder, right? So I guess as long as we keep this instance around, it is fine.
My worry is creating too many clients, since we have to create a new one when our tokens expire (every 30 minutes or so). 30 minutes is more than enough for resetting the socket, so we dont get port exhaustion, but it made me ask anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950105645
Ill make some tests and see what the broker does, and report back.
In my initial tests,the client seems to simply time out.
So my guess is that the broker simply does not respond when you try and send anything to it with a n expired jwt. (Same for a wrong jwt).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950105645
Ill make some tests and see what the broker does, and report back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1028824684
Yeah, go nuts, you just need to implement IAuthentication :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean closed issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean closed issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1028828300
Awesome, I will most likely contribute an OAuth factory within a couple of days then!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1027952975
Hi @VisualBean
In Master, we now have support for on-the-fly authentication. Does that fit your needs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-946131329
Hi @VisualBean. It depends on the usage but I can share some info. The client and its consumers, producers, and readers are all thread-safe. The client has a connection pool and all consumers, producers, and readers accessing the same broker will share the same TCP connection. The consumers, producers, and readers can (with the default exception handling) go into the 'Faulted' state and then you have to create a new consumer/producer/reader.
I hope that helps you with the singleton/scoped/transient questions.
In regards to JWT, DotPulsar just forwards it to the broker, there are currently no plans of creating tokens. Are the other clients supporting that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean edited a comment on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean edited a comment on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-950331145
I trying to figure it out with someone from the pulsar community, when exactly this check happens (the protocol docs aren't very clear on this) - but
> After receiving a Connected response from the broker, the client can consider the connection ready to use. Alternatively, if the broker doesn't validate the client authentication, it will reply with an Error command and close the TCP connection. - https://pulsar.apache.org/docs/en/develop-binary-protocol/#connection-establishment
This doesn't seem to be supported in the dotpulsar client, as I simply got a timeout after waiting a long time. - I will test it again to make sure this is what I am seeing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-947754149
Does the broker close an active connection when the token expires? Or is it just validated when the connection is established and then first becomes a problem when reconnecting due to lost connectivity or something like that?
If the latter, then I think a delegate is a good idea, which will be called each time the client needs to set up a new connection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] VisualBean commented on issue #92: Client usage
Posted by GitBox <gi...@apache.org>.
VisualBean commented on issue #92:
URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-948590185
im not completely sure actually.
The dotpulsar client, will that go into a faulted state when a tcp connection is closed? or will it try and reestablish?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org