You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2017/06/21 19:03:02 UTC
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Alexey Serbin has uploaded a new change for review.
http://gerrit.cloudera.org:8080/7250
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when current
authn token expires. The client automatically retries the RPC that hits
the token expiration error (actually, the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a couple of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
9 files changed, 596 insertions(+), 43 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/1
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#3).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
9 files changed, 656 insertions(+), 66 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/3
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 16: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 16
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 8:
(19 comments)
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS8, Line 229: helper
here and below s/helper/proxy/
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
PS8, Line 276: Preconditions.checkState(state == State.NEGOTIATING);
: Preconditions.checkState
these are assertions not preconditions
PS8, Line 279:
: // Setting state to FAILED_NEGOTIATION.
nit: redundant with the below code
Line 281: // channelDisconnected() or channelClosed() until notified on the authn token re-acquisition results.
high level design question: did you consider actually just throwing an exception on all of the pending RPCs (ie call their errback) but with a specific exception subclass or status code, and then change the error handling/retry logic in AsyncKuduClient to trigger the reacquireAuthnToken() stuff? That seems like it would avoid adding this rather kudu-specific stuff into Connection which seems to regress from the nice improvement you recently made to try to make connections more "KRPC" level constructs.
PS8, Line 289: securityContext.reacquireAuthnToken(this);
seems like it would make more sense to me to call back up into the client here I think?
Line 377: // Explicitly close the channel (regardless whether it's disconnected) and call cleanup().
is this change related? I remember there's a lot of subtlety in this stuff last time I tried to touch it, and makes me nervous to "sneak in" to an unrelated patch
Line 482: return Channels.close(channel);
same here, close vs disconnect can be subtle, not sure why it's in this patch
Line 649: // The object has just been created.
now that you're moving the comments into the appropriate position, mind changing them to /** javadoc style */ ? I think that way they'll show up in IDEs when hovered (not sure if IDEs would show it as is)
PS8, Line 661: negotiation error and there are
: // enqueued messages to handle. Upon failed negotiation event, the Connection object notifies
: // its securityContext to start acquiring a new authn token
this only happens in the case of the specific invalid authn token failure, right? can you clarify, and maybe clarify the state name like AWAITING_NEW_AUTHN_TOKEN or something?
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 68: private final HashMultimap<String, Connection> uuid2connection = HashMultimap.create();
now that this is a multimap, I think it warrants more of a comment explaining what the multiple connections per UUID are
PS8, Line 89: boolean usePrimaryCredentials
nit: javadoc this
and potentially introduce an enum for CredentialsPolicy or something so we get nicer parameter names at call sites?
Line 123: if (foundTerminated) {
don't you want to remove the terminated connection even if you found a non-terminated one above? also agreed with Dan on using an iterator
PS8, Line 140: checkState
isn't this an assert() not a Precondition? ie its failure would indicate a bug in this class's implementation, not a misuse by a caller
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java:
PS8, Line 241: LOG.debug("connection negotiation error: {}", error.getMessage());
can this include the peer address or uuid etc?
PS8, Line 720: // The RPC error received from the server.
nit: use /** ... */
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
File java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java:
Line 47: import com.stumbleupon.async.Callback;
nit: probably goes in the previous section of non-kudu includes
PS8, Line 129: this.tokenReacquirer = new AuthnTokenReacquirer(masterAddresses, masterTable,
: connectToClusterTimeoutMs);
can we avoid the coupling of KuduTable and masterAddresses etc by just taking the AuthnTokenReacquirer as a ctor parameter and constructing it when we create the context? That would also potentially allow easy mocking in the negotiation test, and avoid having the near-duplicate constructor above.
Line 343: private final class AuthnTokenReacquirer {
> Hmm despite what I said I'm not sure this is better than when it was split
hrm, I didn't see the original rev, but I thought it would be less coupling if this were separate as well. I like keeping SecurityContext as a "dumb" object that just stores credentials and related utility code.
PS8, Line 460: /** Notify affected connections on the failure of re-acquire authn token. */
: void notifyAffectedConnections() {
: HashSet<Connection> connections;
: synchronized (affectedConnectionsLock) {
: Preconditions.checkState(!affectedConnections.isEmpty());
: connections = affectedConnections;
: affectedConnections = Sets.newHashSet();
: }
:
: for (final Connection c : connections) {
: c.notifyAuthnTokenAcquired(false);
: }
: }
this code can be shared with above right?
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 668: TO_BE_RENEGOTIATED, // The connection negotiation has failed and has to be be re-negotiated.
> Yep, ideally it would be nice to have just DISCONNECTED state and recycle d
An update: after looking at how to avoid re-connecting a connection, I came into conclusion this might be too cumbersome.
If not allowing the TO_BE_RENEGOTIATED state and further re-connecting, it would require calling status errbacks for all the enqueued RPC messages after a new authentication token is acquired (we cannot silently drop the connection which is in TO_BE_RENEGOTIATED phase because there are enqueued messages). So, this would lead the upper-level code to retry corresponding RPCs, one-at-a-time (basically, that's about finding or establishing a new connection via the connection cache and then creating new copies of the messages in the wire format to be put into the queuedMessages member of other Connection object).
Do we want to go that way?
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 8:
(24 comments)
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS8, Line 229: helper
> here and below s/helper/proxy/
Done
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
PS8, Line 276: Preconditions.checkState(state == State.NEGOTIATING);
: Preconditions.checkState
> these are assertions not preconditions
Is it better to use assert instead of Preconditions here? Could you elaborate why is that, please?
I'm not sure assert is a better choice here since doesn't enforce those checks unless the JVM is run with the '-ea' flag.
PS8, Line 279:
: // Setting state to FAILED_NEGOTIATION.
> nit: redundant with the below code
Done
Line 281: // channelDisconnected() or channelClosed() until notified on the authn token re-acquisition results.
> high level design question: did you consider actually just throwing an exce
Yes, I thought about this. The problem was with keeping the enqueued messages/RPCs somewhere -- I didn't want to do double buffering and have more complex code (the queue in the client level should be per-server). Also, for the time while a new authn token is acquired, it's better to 'plug' particular destination so the client will not try to open and negotiation a new connection in vain, thrashing the connection cache.
I started with the design where it's not necessary to serialize RPCs again, and it was possible to re-connect currently existing connection -- upon re-connection the enqueued wire-format messages would be sent as necessary. But then there was a request to not re-open a connection, so it ended up like this -- some sort of compromise.
If we value clean separation of responsibilities more, then sure it's a way to go. I think it might be the best option -- I'll update this.
PS8, Line 289: securityContext.reacquireAuthnToken(this);
> seems like it would make more sense to me to call back up into the client h
If the re-acquisition logic is moved up, then sure -- it should be done from the higher level.
Line 377: // Explicitly close the channel (regardless whether it's disconnected) and call cleanup().
> is this change related? I remember there's a lot of subtlety in this stuff
Not exactly -- I was thinking making it in the previous changelist (separating Connection).
I just thought it's easier to call Channels.close() regardless of whether the channel is open and then do clean-up from here.
Line 482: return Channels.close(channel);
> same here, close vs disconnect can be subtle, not sure why it's in this pat
Yes, it might. But as I understand if we don't want to keep the channel around, it's better to close right here. Not sure postponing closing the channel buys us anything here. Also, all tests are passing now.
If we are concerned with that, I'm return it back then.
Line 649: // The object has just been created.
> now that you're moving the comments into the appropriate position, mind cha
Done
PS8, Line 661: negotiation error and there are
: // enqueued messages to handle. Upon failed negotiation event, the Connection object notifies
: // its securityContext to start acquiring a new authn token
> this only happens in the case of the specific invalid authn token failure,
Right, renaming into AWAITING_NEW_AUTHN_TOKEN would be better if using current design. However, if switching to the design where the token re-acquisition is started by the upper level, it might make sense to either remove this state or leave this naming.
Line 676: TERMINATED,
> +1 on this name. Definitely makes it more clear that this is the final sta
Yep, I changed it from DISCONNECTED because of that and because FAILED_NEGOTIATION implies the connection is de-facto disconnected already.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 68: private final HashMultimap<String, Connection> uuid2connection = HashMultimap.create();
> now that this is a multimap, I think it warrants more of a comment explaini
Done
PS8, Line 89: boolean usePrimaryCredentials
> nit: javadoc this
Done
Line 114: foundTerminated = true;
> This may be simpler if you use a named iterator to cycle through, and call
ok, this sounds good, thanks.
Line 123: if (foundTerminated) {
> don't you want to remove the terminated connection even if you found a non-
It's more like an optimization -- remove terminated connections when adding new instead of them. Yes, it looks a little bit strange, but number of masters is limited. It will be naturally updated after using to named iterators and doing everything on the first pass.
PS8, Line 140: checkState
> isn't this an assert() not a Precondition? ie its failure would indicate a
Yes, it's more a post-condition. Precondition.checkState() is used here to check for internal state. It might be assert, but asserts are disabled in run-time unless -ea is specified for jvm.
What's wrong with using Preconditions library for consistency checks like this?
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java:
PS8, Line 241: LOG.debug("connection negotiation error: {}", error.getMessage());
> can this include the peer address or uuid etc?
Done
PS8, Line 720: // The RPC error received from the server.
> nit: use /** ... */
Done
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
File java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java:
Line 47: import com.stumbleupon.async.Callback;
> nit: probably goes in the previous section of non-kudu includes
Done
Line 83: * the securityContext expires
> I think this comment may be stale: 'if the current one expires.'
Done
PS8, Line 129: this.tokenReacquirer = new AuthnTokenReacquirer(masterAddresses, masterTable,
: connectToClusterTimeoutMs);
> can we avoid the coupling of KuduTable and masterAddresses etc by just taki
Yup, once AuthnTokenReacquirer is separated back, this will be automatically resolved.
Line 343: private final class AuthnTokenReacquirer {
> hrm, I didn't see the original rev, but I thought it would be less coupling
ok, that makes sense -- I will update this.
The original rev is PS1.
Line 343: private final class AuthnTokenReacquirer {
> Hmm despite what I said I'm not sure this is better than when it was split
ok, np. I will separate it back.
PS8, Line 460: /** Notify affected connections on the failure of re-acquire authn token. */
: void notifyAffectedConnections() {
: HashSet<Connection> connections;
: synchronized (affectedConnectionsLock) {
: Preconditions.checkState(!affectedConnections.isEmpty());
: connections = affectedConnections;
: affectedConnections = Sets.newHashSet();
: }
:
: for (final Connection c : connections) {
: c.notifyAuthnTokenAcquired(false);
: }
: }
> this code can be shared with above right?
Yes. In the original version callback functions were named differently, so it was easier to duplicate this tiny piece of code than instead of trying to re-use.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 93: LOG.error("unexpected exception in test scenario", e);
> Will this cause the test to fail?
Nope, but I'll update it to do so. Thanks!
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has uploaded a new patch set (#11).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
14 files changed, 846 insertions(+), 120 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/11
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#9).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenExpiredException.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
14 files changed, 824 insertions(+), 115 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/9
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 9
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#14).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcTraceFrame.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
15 files changed, 870 insertions(+), 152 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/14
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 14
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 668: TO_BE_RENEGOTIATED, // The connection negotiation has failed and has to be be re-negotiated.
> An update: after looking at how to avoid re-connecting a connection, I came
And another update: I did that. So, Connection to TCP socket is now 1-to-1 relationship (see PS5).
Also, in PS5 all currently established connections are now represented in the connection cache. Prior to PS5 it was possible to have secondary-credentials connection to master be alive and outside of the cache when primary-credentials connection to the same master kicked it out from the cache upon acquiring a new authentication token.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
Line 303: LOG.warn("connect to master: received a new authn token");
> WARN seems excessive for these messages; maybe debug?
hm I'd vote INFO since this would only happen in a very long-running app, where you'd expect occasional INFO messages to be OK.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#16).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcTraceFrame.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
15 files changed, 872 insertions(+), 152 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/16
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 16
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 272: RpcHeader.ErrorStatusPB.RpcErrorCodePB.FATAL_INVALID_AUTHENTICATION_TOKEN)) {
> Could we get in a weird state here where this connection is actually a mast
An update -- the new can be created, but it's not outside of the ConnectionCache. A new connection will be scheduled by tokenReacquirer.reacquire(), but it will be created by using the cache when executing ConnectToCluster.run().
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 1:
(21 comments)
http://gerrit.cloudera.org:8080/#/c/7250/1//COMMIT_MSG
Commit Message:
PS1, Line 9: current
'the current'
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
Line 272: private RpcProxy newRpcProxy(final ServerInfo serverInfo, boolean usePrimaryCredentials) {
Put this directly under the 1-arg version, and copy over the relevant javadoc.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 38: public class AuthnTokenReacquirer {
Did you consider making this functionality part of SecurityContext? Is there an advantage to keeping it separate?
Line 48: private final ReentrantLock lock = new ReentrantLock();
It looks like this could be simplified to just be an Object with synchronized blocks, since the reentrant features aren't used.
Line 54: // TODO(aserbin): remove the client parameter
Looks like this TODO may be stale
Line 65: // The token re-acquirer should be able to mark all the queued messages as failed if after some
Should this be a TODO?
Line 72: * @param connection connection to notify on the
sentence fragment
Line 142: if (e instanceof RecoverableException && retries++ < 5) {
Could you move the increment of retries out of the if condition? It's difficult to determine that it's doing the right thing when it's on the right hand side of a short-circuiting &&.
Line 174: ConnectToCluster.run(masterTable, masterAddresses, null,
This won't reuse cached Master connections, right? Don't we do that on the C++ side?
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 102: private final ClientSocketChannelFactory channelFactory;
Doesn't look like this gets used outside the ctor?
Line 272: if (m instanceof Negotiator.Failure) {
Could we get in a weird state here where this connection is actually a master connection, but not using primary credentials, and this error handling actually spins up a completely new master Connection?
In general, I'm a little uneasy with the possibility that connections are being created outside of the client's connection cache now.
Line 282: Preconditions.checkState(state == State.NEGOTIATING);
Might be worth adding a check that this connection is not using primary credentials
Line 668: TO_BE_RENEGOTIATED, // The connection negotiation has failed and has to be be re-negotiated.
If I understand correctly, this patch is changing Connection so that it actually re-connects after a negotiation failure? Why not keep it simple and keep the 1 TCP Socket to 1 Connection relationship?
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 100: // 3. A connection with primary credentials is requested but currently registered one
Hmm, this comment would seem to contradict my earlier understanding about a Connection now retrying negotiation internally, so I'm a little confused. FWIW this comment is how I would expect things to work.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java:
Line 236: if (header.hasIsError() && header.getIsError()) {
I think this can be simplified to if (header.getIsError()) since the default is false.
Line 240: LOG.debug("connection negotiation error: " + error.getMessage());
Use slf4j string interpolation:
LOG.debug("connection negotiation error: {}" error.getMessage())
Line 716: static class Result {
Perhaps this should be renamed 'Success' to better indicate it's use.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 37: // servers even for not-yet-expired tokens.
.. tablet servers, even for ...
(add comma)
Line 49: c.disconnect();
Kind of ironic that it's called 'getImmutableConnectionsList' :)
Line 62: ListTabletServersResponse response = syncClient.listTabletServers();
Just to ratchet up the intensity a bit, maybe run this test case in a loop and/or with many threads?
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java:
Line 53: FakeDNS.getInstance().install();
What about this setup is different from the other test class? Maybe comment that specifically.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 10:
(20 comments)
looks pretty good, mostly cosmetic comments. I like this design much better
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS10, Line 233: The result proxy object fits for
: * sending RPCs which can be sent over a connection negotiated with secondary credentials
: * (e.g. authn token)
this makes it sound like a restriction.
Really don't you mean that "this doesn't restrict the type of credentials that may be used to connect to the server"? ie if you have Kerberos credentials and no token, it would use those, right?
PS10, Line 235: as for time
: * writing this code, tablet servers do not serve any RPCs which change their behavior depending
: * on the type of authentication credentials.
not sure we need this level of detail about tserver specifics
PS10, Line 260: Preconditions.checkNotNull(serverInfo),
The Guava guide recommends putting Preconditions checks on their own line so it's a little clearer to trace back when they fail: https://github.com/google/guava/wiki/PreconditionsExplained
(they suggest using the inline checkNotNull() result for field initializers)
PS10, Line 1337: Preconditions.checkNotNull
what's the purpose of this? given you are about to dereference it, it would throw an NPE either way (Preconditions.checkNotNull just throws NPE). I think checkNotNull is more useful in cases where you plan to store a reference or otherwise propagate it rather than just immediately dereference it
PS10, Line 1337: Status reasonForRetry
this variable isn't used until down below the if statement -- move it down for better clarity?
Line 1749: return tableId.equals(MASTER_TABLE_NAME_PLACEHOLDER);
this is a purposeful object instance equality check (see the comment above it). Agreed it is weird.
(btw see my earlier comment about unrelated cleanup changes)
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 40: public final class AffectedRpcInfo {
can this be static?
PS10, Line 50: private final AsyncKuduClient client;
: private final SecurityContext securityContext;
: private final List<HostAndPort> masterAddresses;
: private final KuduTable masterTable;
: private final long defaultAdminOperationTimeoutMs;
since the client has all of the other 4 fields inside of it, could we avoid having to take all of these in by making a package-private method like 'client.reconnectToCluster(cb, errb)'?
PS10, Line 111: Preconditions.checkNotNull(rpc), Preconditions.checkNotNull(ex
prefer to see these as the first thing in a function (so it clearly states all the preconditions right at the top)
Line 135: final class NewAuthnTokenCB implements Callback<Void, ConnectToClusterResponse> {
woah, I didn't know you could define a class inside a function. can/should this be static? (same below)
PS10, Line 136: NewAuthnTokenCB() {
: }
remove
PS10, Line 149: success
this gets called even if it didn't get a new token, right?
PS10, Line 173: 5
where'd this constant come from? maybe extract it to a NUM_RETRIES or something?
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 117: * Whether to negotiate the connection using primary credentials; i.e. whether to ignore the
this javadoc sounds like a boolean field. I think just changing to say "Credentials policy to use when authenticating" or something would be fine, since you already have a very nice javadoc on the CredentialsPolicy type itself
PS10, Line 578: Preconditions.checkNotNull(inflightMessages)
found this easier to read non-inline
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
PS10, Line 123: Iterator
shouldn't this be Iterator<Connection> so you don't need the cast below?
PS10, Line 132: credentialsPolicy != Connection.CredentialsPolicy.PRIMARY_CREDENTIALS
I think this would read better as == ANY_CREDENTIALS rather than != PRIMARY
PS10, Line 133: Connection.CredentialsPolicy.PRIMARY_CREDENTIALS
maybe just use 'credentialsPolicy' here?
Line 145: Preconditions.checkState(connections.size() <= 2);
more of an assert IMO
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 99: //noinspection ThrowableResultOfMethodCallIgnored
hm, which lint tool is this suppressing? just curious
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has uploaded a new patch set (#12).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ServerInfo.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
15 files changed, 868 insertions(+), 133 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/12
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 10:
(20 comments)
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS10, Line 233: The result proxy object fits for
: * sending RPCs which can be sent over a connection negotiated with secondary credentials
: * (e.g. authn token)
> this makes it sound like a restriction.
Yes, this comment sounds upside-down and messed up.
Right, if there is no token but only Kerberos credentials, then the Kerberos credentials will be used.
I'll update this accordingly.
PS10, Line 235: as for time
: * writing this code, tablet servers do not serve any RPCs which change their behavior depending
: * on the type of authentication credentials.
> not sure we need this level of detail about tserver specifics
Done
PS10, Line 260: Preconditions.checkNotNull(serverInfo),
> The Guava guide recommends putting Preconditions checks on their own line s
Indeed -- using checkNotNull() would not make much sense here since it's the same NPE, but coming just from this particular place instead of the point of actual de-reference.
I'll remove it from here and other similar places.
PS10, Line 1337: Status reasonForRetry
> this variable isn't used until down below the if statement -- move it down
Done
PS10, Line 1337: Preconditions.checkNotNull
> what's the purpose of this? given you are about to dereference it, it would
Agreed -- I rather mechanically placed Preconditions everywhere in the new code. I will update this and other places.
Line 1749: return tableId.equals(MASTER_TABLE_NAME_PLACEHOLDER);
> this is a purposeful object instance equality check (see the comment above
I looked at it and this seems safe since this method is private.
Agreed that sneaking unrelated changes into bigger changelists is not a good idea.
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 40: public final class AffectedRpcInfo {
> can this be static?
Done
PS10, Line 50: private final AsyncKuduClient client;
: private final SecurityContext securityContext;
: private final List<HostAndPort> masterAddresses;
: private final KuduTable masterTable;
: private final long defaultAdminOperationTimeoutMs;
> since the client has all of the other 4 fields inside of it, could we avoid
It's a great idea.
PS10, Line 111: Preconditions.checkNotNull(rpc), Preconditions.checkNotNull(ex
> prefer to see these as the first thing in a function (so it clearly states
Yep, but then the linter complains.
I think it's better to move the non-null precondition check into the AffectedRpcInfo constructor, as discussed somewhere else.
Line 135: final class NewAuthnTokenCB implements Callback<Void, ConnectToClusterResponse> {
> woah, I didn't know you could define a class inside a function. can/should
I tried to make it static but the compiler wasn't happy about that:
Error:(136, 18) java: modifier static not allowed here
After some research, I found more info where this restriction is stated explicitly:
http://docs.oracle.com/javase/tutorial/java/javaOO/localclasses.html
https://docstore.mik.ua/orelly/java-ent/jnut/ch03_11.htm
PS10, Line 136: NewAuthnTokenCB() {
: }
> remove
Done
PS10, Line 149: success
> this gets called even if it didn't get a new token, right?
Right -- I updated the comment and the code in the call() method as well.
PS10, Line 173: 5
> where'd this constant come from? maybe extract it to a NUM_RETRIES or somet
Done
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 117: * Whether to negotiate the connection using primary credentials; i.e. whether to ignore the
> this javadoc sounds like a boolean field. I think just changing to say "Cre
Done
PS10, Line 578: Preconditions.checkNotNull(inflightMessages)
> found this easier to read non-inline
I removed it as per the discussion: use the checkNotNull only when storing the references for future access.
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
PS10, Line 123: Iterator
> shouldn't this be Iterator<Connection> so you don't need the cast below?
Done
PS10, Line 132: credentialsPolicy != Connection.CredentialsPolicy.PRIMARY_CREDENTIALS
> I think this would read better as == ANY_CREDENTIALS rather than != PRIMARY
Done
PS10, Line 133: Connection.CredentialsPolicy.PRIMARY_CREDENTIALS
> maybe just use 'credentialsPolicy' here?
yep, and that's better for the readability.
Line 145: Preconditions.checkState(connections.size() <= 2);
> more of an assert IMO
Done
http://gerrit.cloudera.org:8080/#/c/7250/10/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 99: //noinspection ThrowableResultOfMethodCallIgnored
> hm, which lint tool is this suppressing? just curious
This is about returning a Throwable instance without throwing it out (HashMap.put() returns Throwable). It's the suppression for IntelliJ Idea code inspector: Analyze > Inspect Code...).
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 14:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/7250/14/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
Line 1140: ? new NonRecoverableException(statusTimedOut)
From a quick look, it seems like a null cause will be passed all the way up the chain without being dereferenced, so I don't think the ternary is necessary.
Line 1382:
extra space
http://gerrit.cloudera.org:8080/#/c/7250/14/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 33: final class AuthnTokenReacquirer {
Add visibility/privacy annotations.
Line 38: private final Object affectedRpcInfoListLock = new Object();
Make the name of this match the list (there's an extra 'info' currently).
Line 46: private ArrayList<KuduRpc<?>> affectedRpcList = Lists.newArrayList();
I think this would be better called 'queuedRpcs' or 'rpcsAwaitingToken'. Affected is a little bit generic. I prefer the first because it make the desired FIFO semantics explicit.
Line 82: rpc.addTrace(new RpcTraceFrame.RpcTraceFrameBuilder(
I was thinking that this trace frame should be added when the RPC is popped, so that the timestamp includes the time waiting in the queue, but I'm not entirely sure. I think we should get JD to weigh in.
Line 93: private List<KuduRpc<?>> prepareRpcListToNotify() {
This naming can be improved. The class is using a standard double-buffering technique, but the names and (lack of) comments don't make that clear. I'd suggest 'swapQueuedRpcs'.
Line 96: Preconditions.checkState(!affectedRpcList.isEmpty());
Move this after the synchronized block (and therefore on the local instead of the field), and considering making this an 'assert', since the method is private and this would indicate a logic bug in the class.
Line 109: @Override
Put annotation after comments.
Line 128: void handleAffectedRpcs() {
naming: 'retryQueuedRpcs' would be more indicative of what is happening.
PS14, Line 165: handleAffectedRpcs
naming: 'failQueuedRpcs'
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 14
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#5).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
12 files changed, 744 insertions(+), 102 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/5
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 155: this.attempts = 0;
this is now redundant with the field initializer
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/ServerInfo.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ServerInfo.java:
Line 36: @Nonnull
nit: again would prefer not to have these random changes in this commit. I'm not gonna force you go to back and remove them, but it really makes reviewing harder when there are a bunch of changes like this (plus cherry-picking becomes very hard to resolve conflicts, etc)
Unless this commit is actually changing semantics here such that we used to sometimes have these be nullable but now they are never null, or whatever. Seeing this change adds cognitive load for the reviewer thinking "ah, what is this patch changing about how ServerInfo is used/constructed/etc?" and distracts from the meat of what's actually changing.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
> (2 comments)
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(16 comments)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS12, Line 234: Kerberos credentials
It may be more accurate to say 'SASL' credentials, since it will be Kerberos if available, or PLAIN otherwise.
PS12, Line 297: Kerberos credentials
Here as well.
Line 303: LOG.warn("connect to master: received a new authn token");
WARN seems excessive for these messages; maybe debug?
Line 305: cb.call(Boolean.TRUE);
Can it just pass true/false here and below? it should auto-box.
PS12, Line 307: connecto
connect
Line 344: AuthnTokenReacquirer getTokenReacquirer() {
@VisibleForTesting?
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 37: public static final class AffectedRpcInfo {
Shouldn't the fact that an RPC got delayed waiting for a token go into it's trace? Why the need to also keep around the exception separately?
I think the RPC should never fail with the same exception; the only thing that can happen after being popped from this queue are a timeout, or a retry (which will have it's own exception if it fails).
Line 108:
extra line
PS12, Line 119: attemp
attempt
PS12, Line 119: uccessu
succesful
PS12, Line 122: {@link Boolean.TRUE}
These can just be {@code true} / {@code false}
Line 125: public Void call(Boolean tokenAcquired) throws Exception {
tokenAcquired isn't used?
PS12, Line 133: Kerberos
primary
Line 145: * Errback to retry authn token re-acquisition and notify the handle the affected RPCs if the
Perhaps we should retry indefinitely with increasing backoff, but aggressively timeout RPCs in the queue after each failure?
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 669: NEGOTIATION_FAILED,
The control flow around NEGOTIATION_FAILED is non-local, and hard to follow. If I understand correctly, we receive a message indicating that negotiation failed inside messageReceived, then we shove the failure condition into a field, and then wait for cleanup() to be called, wherin we do something special because that field is set. Could this be simplified by doing the special negotiation failed cleanup directly in messageReceived, then going directly to TERMINATED state? That might simplify the non-local control flow, and remove one of these states.
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 69: private final ReentrantLock lock = new ReentrantLock();
It appears this no longer needs the re-entrant features, so it could be a normal Object with synchronized blocks (or even synchronize on uuid2connection directly).
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 11: Verified+1
unrelated flake in RaftConsensusITest.TestChurnyElections_WithDuplicateKeys
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
PS8, Line 276: Preconditions.checkState(state == State.NEGOTIATING);
: Preconditions.checkState
> true that assertions aren't on by default, so they are more like DCHECK() i
Probably.
My main motivation behind using Preconditions vs assert is because in some cases it's a situation which cannot be handled consistently if continuing further when a precondition is not true. Assert would just ignore that in non-debug run, and I'm not sure that's what we want.
Line 482: return Channels.close(channel);
> I'm not particularly concerned with the change, but generally would rather
Agreed -- it's not a good idea to unintentionally 'hide' such small modifications in other changes.
As I understand, the difference with disconnect() and close() is that closed channel cannot be re-used (i.e. open) again. And from the functionality point it seems both disconnect() and close() do the same in cast of TCP connections:
https://stackoverflow.com/questions/14128910/netty-calling-channel-disconnect-actually-closes-the-channel
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#4).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
9 files changed, 656 insertions(+), 66 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/4
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 8:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 676: TERMINATED,
+1 on this name. Definitely makes it more clear that this is the final state, and it can't be changed after this.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 114: foundTerminated = true;
This may be simpler if you use a named iterator to cycle through, and call remove() if the connection is terminated. That way you don't need a flag / have to do another loop through.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
File java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java:
Line 83: * the securityContext expires
I think this comment may be stale: 'if the current one expires.'
Line 343: private final class AuthnTokenReacquirer {
Hmm despite what I said I'm not sure this is better than when it was split out. I was more imagining the functionality would be entirely subsumed into SecurityContext (just a single class), but maybe that's too much for one class. The test-only ctor of the parent class is definitely a smell.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 93: LOG.error("unexpected exception in test scenario", e);
Will this cause the test to fail?
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#13).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
14 files changed, 873 insertions(+), 147 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/13
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 13
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 1:
(21 comments)
Thank you for the review. I'll address the major points in the next patch revision. Right now I wanted to quickly respond and bring the code up-to-date with the updated revision of the parent patch.
http://gerrit.cloudera.org:8080/#/c/7250/1//COMMIT_MSG
Commit Message:
PS1, Line 9: current
> 'the current'
Done
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
Line 272: private RpcProxy newRpcProxy(final ServerInfo serverInfo, boolean usePrimaryCredentials) {
> Put this directly under the 1-arg version, and copy over the relevant javad
Done
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 38: public class AuthnTokenReacquirer {
> Did you consider making this functionality part of SecurityContext? Is the
Nope, I didn't think about making this is a part of SecurityContext. I'll give it a try, thanks.
Line 48: private final ReentrantLock lock = new ReentrantLock();
> It looks like this could be simplified to just be an Object with synchroniz
This sounds good, I will give it a try.
Line 54: // TODO(aserbin): remove the client parameter
> Looks like this TODO may be stale
Done
Line 65: // The token re-acquirer should be able to mark all the queued messages as failed if after some
> Should this be a TODO?
Right -- that was supposed to be a TODO. Done.
Line 72: * @param connection connection to notify on the
> sentence fragment
Done
Line 142: if (e instanceof RecoverableException && retries++ < 5) {
> Could you move the increment of retries out of the if condition? It's diff
Done
Line 174: ConnectToCluster.run(masterTable, masterAddresses, null,
> This won't reuse cached Master connections, right? Don't we do that on the
This will reuse cached Master connection if the connection to the requested destination is present in the cache and established with primary credentials.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 102: private final ClientSocketChannelFactory channelFactory;
> Doesn't look like this gets used outside the ctor?
Right -- done.
Line 272: if (m instanceof Negotiator.Failure) {
> Could we get in a weird state here where this connection is actually a mast
Yep, that's possible -- I replied on your comment on ConnectionCache.getConnection() method.
Line 282: Preconditions.checkState(state == State.NEGOTIATING);
> Might be worth adding a check that this connection is not using primary cre
That's a good idea. Yes, if that happens, we should report an exception so it would be handled appropriately.
Line 668: TO_BE_RENEGOTIATED, // The connection negotiation has failed and has to be be re-negotiated.
> If I understand correctly, this patch is changing Connection so that it act
Yep, ideally it would be nice to have just DISCONNECTED state and recycle disconnected connections. But then it's necessary to move the queued RPCs into a new connection. Probably, that can be done at the ConnectionCache level. I'll take a closer look at that.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 100: // 3. A connection with primary credentials is requested but currently registered one
> Hmm, this comment would seem to contradict my earlier understanding about a
Probably, your confusion comes from the naming for the 'DISCONNECTED' state. One small detail: when connection negotiation fails, the state changes to TO_BE_RENEGOTIATED, not DISCONNECTED. The DISCONNECTED state is a terminal state in the state diagram.
And yes, a Connection object starts re-negotiation, after getting into TO_BE_RENEGOTIATED state. Such a connection will stay in the cache unless that's a connection to the master where we need to establish a new connection using primary credentials. In the latter case, the re-negotiation happens anyway, but the re-negotiated connection is replaced in the cache with the primary-credentials-connection-to-master which is used to re-acquire authn token. The queued RPCs from the re-negotiated old connection-to-the-same-master will be sent when the connection is re-negotiated using the new authn token. In that case there will be two connections from the client to the same master. I think it's not a big issue because there might be extra M connections at most, where M is number of master servers in the cluster (which is limited).
A 'cleaner way' would be moving the queued RPCs from the old connection-to-the-same-master into the newly established connection once an authn token is acquired. However, there might be a race of closing the newly established connection and assigning queued RPCs from the old connection to the newly established one. One way of addressing that race would be allowing a connection in the DISCONNECTED state to re-connect. I didn't like that idea.
However, there is another way of addressing that -- handling the queued RPCs using re-acquirer itself. I'll address that.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java:
Line 236: if (header.hasIsError() && header.getIsError()) {
> I think this can be simplified to if (header.getIsError()) since the defaul
Done
Line 240: LOG.debug("connection negotiation error: " + error.getMessage());
> Use slf4j string interpolation:
Done
Line 716: static class Result {
> Perhaps this should be renamed 'Success' to better indicate it's use.
Done
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java:
Line 37: // servers even for not-yet-expired tokens.
> .. tablet servers, even for ...
Done
Line 49: c.disconnect();
> Kind of ironic that it's called 'getImmutableConnectionsList' :)
Yep, that's an inconsistent naming -- agreed. I tried to preserve the original naming.
As I understand, the idea was to reflect the fact the _list_ is immutable, i.e. removing/adding connections from/to the result container does not affect the actual set of client's connections.
I'll rename it.
Line 62: ListTabletServersResponse response = syncClient.listTabletServers();
> Just to ratchet up the intensity a bit, maybe run this test case in a loop
That's a good idea. Done.
http://gerrit.cloudera.org:8080/#/c/7250/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java:
Line 53: FakeDNS.getInstance().install();
> What about this setup is different from the other test class? Maybe commen
Here we want to have just 1 master to simplify the logic of the test.
I will add a comment.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 5: Verified+1
Unrelated flake in RaftConsensusITest.TestReplicaBehaviorViaRPC
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#15).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcTraceFrame.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
15 files changed, 867 insertions(+), 152 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/15
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 15
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 37: public static final class AffectedRpcInfo {
> I keep the exception due to the 'technical' reasons: it's needed to retry t
Right, so I'm worried this is actually going to distort the traces. First off, the timestamp is no longer going to correspond to the actual failure time. I think we may want to add a new trace state ('queued waiting for token acquire' or something), so that we can see that in RPC traces, and marked with timestamp of when it's popped from the queue. As far as having to pass the exception to handleRetryableErrorNoDelay, isn't that a method that is only called here? Seems fixable.
Line 145: * Errback to retry authn token re-acquisition and notify the handle the affected RPCs if the
> That's a good idea. What if I just it just a TODO here for now?
That's fine with me.
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 669: NEGOTIATION_FAILED,
> Yes, we could do that. But
If possible it would be nice to do in this commit, because this commit is introducing the state. If it proves to be more complex I'm fine skipping it though (but I suspect it will make the control flow simpler).
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 9: Verified+1
Unrelated flake in MultiThreadedTabletTest/1.DeleteAndReinsert
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 9
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 10: Verified+1
unrelated flake in master_replication-itest
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has uploaded a new patch set (#10).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
14 files changed, 836 insertions(+), 116 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/10
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#2).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/test/java/org/apache/kudu/client/ITClient.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
13 files changed, 676 insertions(+), 78 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/2
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7250
to look at the new patch set (#8).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
12 files changed, 717 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/8
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 10:
(2 comments)
just leaving some reponses to the comments on r8. will look at the new rev following.
http://gerrit.cloudera.org:8080/#/c/7250/8/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
PS8, Line 276: euedMessages = null;
: // Send out all the enqueu
> Is it better to use assert instead of Preconditions here? Could you elabor
true that assertions aren't on by default, so they are more like DCHECK() in our C code. I believe we do enable them in our unit test runs, at least.
I dislike Preconditions here because the type of exception thrown is "IllegalStateException" which is supposed to indicate to a caller that they called a method at an inappropriate time. In my mind it's meant to indicate "misuse" of an API, rather than "mis-programming/bug" of an API (which an AssertionError more clearly represents).
Again to make the comparison to our C++ style, we use CHECK/DCHECK for cases where it would be an internal bug to see it fail, and "if (...) return Status::IllegalArgument(...)" or whatever when it would just be misuse of an API.
Perhaps worth starting a thread about this so we can bike-shed an appropriate amount?
Line 482:
> Yes, it might. But as I understand if we don't want to keep the channel ar
I'm not particularly concerned with the change, but generally would rather such things not be "hidden" in a larger patch that doesn't really relate. It's easy to miss a small one-line change like this when reviewing a big patch, but the small change may actually require as much thinking/review as the remaining hundreds, when it's something subtle :)
(totally OK with small "cleanup" like fixing formatting or other trivial stuff, but at least I dont really know the difference between Netty channel close vs disconnect without having to go spend 10-15 minutes reading Netty docs or source)
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 37: public static final class AffectedRpcInfo {
> Right, so I'm worried this is actually going to distort the traces. First
Good catch! Originally I used that delayed RPC retry method and that would keep the traces consistent automatically (but not having right timestamp, which I thought to ignore).
Now it's just another reason to do it righ. Thanks!
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 669: NEGOTIATION_FAILED,
> If possible it would be nice to do in this commit, because this commit is i
I would prefer to publish it separately, since I'm sure it will work. At least, my first attempt to simply call cleanup() from the handler failed.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
Patch Set 12:
(19 comments)
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
PS12, Line 234: Kerberos credentials
> It may be more accurate to say 'SASL' credentials, since it will be Kerbero
Done
PS12, Line 297: Kerberos credentials
> Here as well.
Done
Line 303: LOG.warn("connect to master: received a new authn token");
> WARN seems excessive for these messages; maybe debug?
Done
Line 303: LOG.warn("connect to master: received a new authn token");
> hm I'd vote INFO since this would only happen in a very long-running app, w
Done
Line 305: cb.call(Boolean.TRUE);
> Can it just pass true/false here and below? it should auto-box.
Done
PS12, Line 307: connecto
> connect
Done
Line 344: AuthnTokenReacquirer getTokenReacquirer() {
> @VisibleForTesting?
I updates the related code to be like 'handleRetriableRpc()' and other methods, so this accessor is not longer necessary. Thank you for pointing at this.
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java:
Line 37: public static final class AffectedRpcInfo {
> Shouldn't the fact that an RPC got delayed waiting for a token go into it's
I keep the exception due to the 'technical' reasons: it's needed to retry the RPC using existing signature: it's needed to add the information about the retry into the trace.
Line 108:
> extra line
Done
PS12, Line 119: uccessu
> succesful
Done
PS12, Line 119: attemp
> attempt
Done
PS12, Line 122: {@link Boolean.TRUE}
> These can just be {@code true} / {@code false}
Done
Line 125: public Void call(Boolean tokenAcquired) throws Exception {
> tokenAcquired isn't used?
yep, here it's not used. I'm not sure what to do in this case - just retry the operation? From the other case, the case when we got no token back and no exception detected manifests a problem on the server side.
PS12, Line 133: Kerberos
> primary
Done
Line 145: * Errback to retry authn token re-acquisition and notify the handle the affected RPCs if the
> Perhaps we should retry indefinitely with increasing backoff, but aggressiv
That's a good idea. What if I just it just a TODO here for now?
Line 155: this.attempts = 0;
> this is now redundant with the field initializer
Done
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
File java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java:
Line 669: NEGOTIATION_FAILED,
> The control flow around NEGOTIATION_FAILED is non-local, and hard to follow
Yes, we could do that. But
1) I'm not sure it's safe to do it right here, in the handler
2) Failed negotiation is a special case which requires some special handling.
Do you mind if I try to address this in a follow-up commit?
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java:
Line 69: private final ReentrantLock lock = new ReentrantLock();
> It appears this no longer needs the re-entrant features, so it could be a n
It's a good observation. Done.
http://gerrit.cloudera.org:8080/#/c/7250/12/java/kudu-client/src/main/java/org/apache/kudu/client/ServerInfo.java
File java/kudu-client/src/main/java/org/apache/kudu/client/ServerInfo.java:
Line 36: @Nonnull
> nit: again would prefer not to have these random changes in this commit. I'
Sorry about that -- reverted.
I agree it's better to put it separate unless it's really necessary in the changelist.
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged.
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Reviewed-on: http://gerrit.cloudera.org:8080/7250
Tested-by: Kudu Jenkins
Reviewed-by: Dan Burkert <da...@apache.org>
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
A java/kudu-client/src/main/java/org/apache/kudu/client/AuthnTokenReacquirer.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
A java/kudu-client/src/main/java/org/apache/kudu/client/InvalidAuthnTokenException.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcTraceFrame.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
15 files changed, 872 insertions(+), 152 deletions(-)
Approvals:
Dan Burkert: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 17
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] KUDU-2013 re-acquire authn token if expired
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has uploaded a new patch set (#6).
Change subject: [java] KUDU-2013 re-acquire authn token if expired
......................................................................
[java] KUDU-2013 re-acquire authn token if expired
This patch introduces automatic authn token re-acquisition when the
current authn token expires. The client automatically retries the RPC
that hits the token expiration error (the error to re-try is seen as
FATAL_INVALID_AUTHENTICATION_TOKEN sent by the server during connection
negotiation).
Added a few of tests to exercise the new retry logic for automatic
token re-acquisition in case of master-only operations, a bare minimum
workload scenario, and one special case of a connection to the master
opened with secondary credentials.
Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquire.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestConnectionCache.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java
12 files changed, 727 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7250/6
--
To view, visit http://gerrit.cloudera.org:8080/7250
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0be620629c9a8345ecd5e5679c80ee76ca4eaa57
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>