You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2017/03/08 00:34:37 UTC
[kudu-CR](branch-1.3.x) java: avoid spewing ClosedChannelException on client-initiated disconnects
Todd Lipcon has uploaded a new change for review.
http://gerrit.cloudera.org:8080/6305
Change subject: java: avoid spewing ClosedChannelException on client-initiated disconnects
......................................................................
java: avoid spewing ClosedChannelException on client-initiated disconnects
When submitting an MR job, the submitting program opens a client
instance for a short amount of time, just to fetch an authentication
token. When it then calls 'close()', any TabletClient connections that
the client may still have open get disconnected, and that was causing an
exception to be logged like follows:
17/03/07 13:44:51 ERROR client.TabletClient: [Peer ] Unexpected exception from downstream on [id: 0x06a6a87b, /172.31.112.110:41188 :> kudu-security-1.gce.cloudera.com/172.31.112.110:7051]
java.nio.channels.ClosedChannelException
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:433)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:128)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:84)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:725)
at org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
at org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:704)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:671)
at org.apache.kudu.client.Negotiator.sendSaslMessage(Negotiator.java:218)
at org.apache.kudu.client.Negotiator.sendTunneledTls(Negotiator.java:515)
at org.apache.kudu.client.Negotiator.sendPendingOutboundTls(Negotiator.java:505)
at org.apache.kudu.client.Negotiator.handleTlsMessage(Negotiator.java:451)
at org.apache.kudu.client.Negotiator.handleResponse(Negotiator.java:250)
at org.apache.kudu.client.Negotiator.messageReceived(Negotiator.java:229)
This patch adds a flag in TabletClient which keeps track of the fact that
the disconnection was requested rather than unexpected, and in that case
avoids logging anything.
A new test triggers the same connection, captures logs, and makes sure
that no exceptions are in the logs.
Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Reviewed-on: http://gerrit.cloudera.org:8080/6303
Reviewed-by: Jean-Daniel Cryans <jd...@apache.org>
Tested-by: Kudu Jenkins
(cherry picked from commit 5566bc902ef94a9ce825f7c2f939cf87428941df)
---
M java/kudu-client/src/main/java/org/apache/kudu/client/TabletClient.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/util/CapturingLogAppender.java
3 files changed, 117 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/6305/1
--
To view, visit http://gerrit.cloudera.org:8080/6305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Todd Lipcon <to...@apache.org>
[kudu-CR](branch-1.3.x) java: avoid spewing ClosedChannelException on client-initiated disconnects
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: java: avoid spewing ClosedChannelException on client-initiated disconnects
......................................................................
Patch Set 2:
Yea, that test failure is tracked in a JIRA somewhere
--
To view, visit http://gerrit.cloudera.org:8080/6305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR](branch-1.3.x) java: avoid spewing ClosedChannelException on client-initiated disconnects
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change.
Change subject: java: avoid spewing ClosedChannelException on client-initiated disconnects
......................................................................
Patch Set 1: Code-Review+2 Verified+1
Test failure is not related but it's really scary:
delta_tracker.cc:208] Check failed: first_copy->delta_stats().min_timestamp() >= second_copy->delta_stats().min_timestamp() (10367 vs. 10375) Found out-of-order deltas: [{11285962673355902900 (ts range=[10367, 10571], delete_count=[0], reinsert_count=[0], update_counts_by_col_id=[12:5))}, {11285962673355902887 (ts range=[10375, 10402], delete_count=[0], reinsert_count=[0], update_counts_by_col_id=[12:8))}]: type = 1
--
To view, visit http://gerrit.cloudera.org:8080/6305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-HasComments: No
[kudu-CR](branch-1.3.x) java: avoid spewing ClosedChannelException on client-initiated disconnects
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has submitted this change and it was merged.
Change subject: java: avoid spewing ClosedChannelException on client-initiated disconnects
......................................................................
java: avoid spewing ClosedChannelException on client-initiated disconnects
When submitting an MR job, the submitting program opens a client
instance for a short amount of time, just to fetch an authentication
token. When it then calls 'close()', any TabletClient connections that
the client may still have open get disconnected, and that was causing an
exception to be logged like follows:
17/03/07 13:44:51 ERROR client.TabletClient: [Peer ] Unexpected exception from downstream on [id: 0x06a6a87b, /172.31.112.110:41188 :> kudu-security-1.gce.cloudera.com/172.31.112.110:7051]
java.nio.channels.ClosedChannelException
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:433)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:128)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:84)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:725)
at org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
at org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:704)
at org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:671)
at org.apache.kudu.client.Negotiator.sendSaslMessage(Negotiator.java:218)
at org.apache.kudu.client.Negotiator.sendTunneledTls(Negotiator.java:515)
at org.apache.kudu.client.Negotiator.sendPendingOutboundTls(Negotiator.java:505)
at org.apache.kudu.client.Negotiator.handleTlsMessage(Negotiator.java:451)
at org.apache.kudu.client.Negotiator.handleResponse(Negotiator.java:250)
at org.apache.kudu.client.Negotiator.messageReceived(Negotiator.java:229)
This patch adds a flag in TabletClient which keeps track of the fact that
the disconnection was requested rather than unexpected, and in that case
avoids logging anything.
A new test triggers the same connection, captures logs, and makes sure
that no exceptions are in the logs.
Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Reviewed-on: http://gerrit.cloudera.org:8080/6303
Reviewed-by: Jean-Daniel Cryans <jd...@apache.org>
Tested-by: Kudu Jenkins
(cherry picked from commit 5566bc902ef94a9ce825f7c2f939cf87428941df)
Reviewed-on: http://gerrit.cloudera.org:8080/6305
Tested-by: Jean-Daniel Cryans <jd...@apache.org>
---
M java/kudu-client/src/main/java/org/apache/kudu/client/TabletClient.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
A java/kudu-client/src/test/java/org/apache/kudu/util/CapturingLogAppender.java
3 files changed, 117 insertions(+), 0 deletions(-)
Approvals:
Jean-Daniel Cryans: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/6305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4e940d821c7d3f670c5a6b7407385952dc9debfc
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>