You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by iv...@apache.org on 2017/12/14 22:31:27 UTC
[bookkeeper] 02/02: Race in PCBC connection if channel inactive
before callback
This is an automated email from the ASF dual-hosted git repository.
ivank pushed a commit to branch branch-4.6
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
commit 9c7079a82ee64520e1056652216a445a7531668f
Author: Ivan Kelly <iv...@apache.org>
AuthorDate: Mon Dec 11 21:57:51 2017 +0100
Race in PCBC connection if channel inactive before callback
There's a race in PCBC where if the channel is errored before the
connection callback occurs, the channel will hang around as a zombie
forever.
This caused a flake in
TestDisableEnsembleChange#testRetryFailureBookie, but the problem
itself could affect production code. This test hit it because it tries
to reconnect in a tight loop as a bookie is booting.
The error occurs because we only change state with channelInactive if
the channelInactive occurred on the channel that matches the currently
assigned this.channel. If the channel connects successfully, but
channelInactive occurs before the connect callback runs, the channel
will be closed, but the state will be transitioned to CONNECTED. No
more events will occur on the channel, so it'll hang around forever,
and all requests on the channel will timeout.
The fix is to check that the channel is active before assigning it to
this.channel and setting state to CONNECTED. If the channel is not
active at this point, then the else clause of this block will run
which closes the channel, and sets state to DISCONNECTED. In this
case, the next request will trigger a new connection attempt.
Author: Ivan Kelly <iv...@apache.org>
Reviewers: Matteo Merli <mm...@apache.org>
This closes #827 from ivankelly/ensemble-flake
---
.../main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
index c392c79..3d9ba13 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
@@ -1851,7 +1851,7 @@ public class PerChannelBookieClient extends ChannelInboundHandlerAdapter {
Queue<GenericCallback<PerChannelBookieClient>> oldPendingOps;
synchronized (PerChannelBookieClient.this) {
- if (future.isSuccess() && state == ConnectionState.CONNECTING) {
+ if (future.isSuccess() && state == ConnectionState.CONNECTING && future.channel().isActive()) {
LOG.info("Successfully connected to bookie: {}", future.channel());
rc = BKException.Code.OK;
channel = future.channel();
--
To stop receiving notification emails like this one, please contact
"commits@bookkeeper.apache.org" <co...@bookkeeper.apache.org>.