You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2017/04/19 02:55:46 UTC

[kudu-CR] [rpc-test] fixed TestConnectionKeepalive flakiness

Alexey Serbin has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/6683

Change subject: [rpc-test] fixed TestConnectionKeepalive flakiness
......................................................................

[rpc-test] fixed TestConnectionKeepalive flakiness

After introducing TLS option for Kudu RPC, the TestConnectionKeepalive
became a bit flaky, failing time to time with output like the following:

[ RUN      ] OptionalSSL/TestRpc.TestConnectionKeepalive/1
I0419 01:31:19.019090  4863 rpc-test.cc:223] Connecting to 0.0.0.0:52055
src/kudu/rpc/rpc-test.cc:225: Failure
Failed
Bad status: Network error: Recv() got EOF from remote (error 108)

[ RUN      ] OptionalSSL/TestRpc.TestConnectionKeepalive/1
I0419 01:31:25.609285   370 rpc-test.cc:223] Connecting to 0.0.0.0:41418
W0419 01:31:25.853305   398 connection.cc:462] client connection to \
  0.0.0.0:41418 recv error: Network error: recv error: \
  Connection reset by peer (error 104)
src/kudu/rpc/rpc-test.cc:225: Failure
Failed
Bad status: Network error: recv error: Connection reset by peer (error 104)

It seems the TLS connection establishment phase in some cases takes too long,
so by the time of test RPC call might close.

Bumping the keepalive interval helped to address the issue.  Prior to
this fix, if running with --stress_cpu_threads=32 option,
usually 15 out of 1024 runs failed.  After the fix 0 out of 4096 runs failed.

Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
---
M src/kudu/rpc/rpc-test.cc
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/83/6683/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>

[kudu-CR] [rpc-test] fixed TestConnectionKeepalive flakiness

Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.

Change subject: [rpc-test] fixed TestConnectionKeepalive flakiness
......................................................................


Patch Set 1: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/6683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No

[kudu-CR] [rpc-test] fixed TestConnectionKeepalive flakiness

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged.

Change subject: [rpc-test] fixed TestConnectionKeepalive flakiness
......................................................................


[rpc-test] fixed TestConnectionKeepalive flakiness

After introducing TLS option for Kudu RPC, the TestConnectionKeepalive
became a bit flaky, failing time to time with output like the following:

[ RUN      ] OptionalSSL/TestRpc.TestConnectionKeepalive/1
I0419 01:31:19.019090  4863 rpc-test.cc:223] Connecting to 0.0.0.0:52055
src/kudu/rpc/rpc-test.cc:225: Failure
Failed
Bad status: Network error: Recv() got EOF from remote (error 108)

[ RUN      ] OptionalSSL/TestRpc.TestConnectionKeepalive/1
I0419 01:31:25.609285   370 rpc-test.cc:223] Connecting to 0.0.0.0:41418
W0419 01:31:25.853305   398 connection.cc:462] client connection to \
  0.0.0.0:41418 recv error: Network error: recv error: \
  Connection reset by peer (error 104)
src/kudu/rpc/rpc-test.cc:225: Failure
Failed
Bad status: Network error: recv error: Connection reset by peer (error 104)

It seems the TLS connection establishment phase in some cases takes too long,
so by the time of test RPC call might close.

Bumping the keepalive interval helped to address the issue.  Prior to
this fix, if running with --stress_cpu_threads=32 option,
usually 15 out of 1024 runs failed.  After the fix 0 out of 4096 runs failed.

Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
Reviewed-on: http://gerrit.cloudera.org:8080/6683
Tested-by: Kudu Jenkins
Reviewed-by: Sailesh Mukil <sa...@cloudera.com>
Reviewed-by: Dan Burkert <da...@apache.org>
---
M src/kudu/rpc/rpc-test.cc
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Sailesh Mukil: Looks good to me, but someone else must approve
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/6683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] [rpc-test] fixed TestConnectionKeepalive flakiness

Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change.

Change subject: [rpc-test] fixed TestConnectionKeepalive flakiness
......................................................................


Patch Set 1: Code-Review+1

LGTM

-- 
To view, visit http://gerrit.cloudera.org:8080/6683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8fb2dec953c1047ea43fe9e28d470ae6566077c5
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No