[kudu-CR] itests: address some multi-master flakiness

["Adar Dembo (Code Review)" <ge...@cloudera.org>]

Hello Todd Lipcon,

I'd like you to do a code review.  Please visit

    http://gerrit.cloudera.org:8080/2368

to review the following change.

Change subject: itests: address some multi-master flakiness
......................................................................

itests: address some multi-master flakiness

As part of the work to ensure multi-master works properly, I did a pass over
the mm integration tests and their flaky failures. Here's what I found.

Many tests are at least a little flaky due to KUDU-1358, as there's really
nothing preventing a master leader election from taking place at any time,
including during a create table setting up a test.

master_failover-itest.cc
- TestCreateTableSync must remain disabled due to KUDU-1358.
- TestPauseAfterCreateTableIssued was disabled from day one (commit
  6be4c23). I suspect this was due to poor deadline handling that has since
  been fixed so I'm reenabling it. It's survived 1000 TSAN runs on ve0518.
- TestDeleteTableSync is flaky due to timeouts in DeleteTable(). Elsewhere
  we use 90s timeouts, so I've made a change to do the same here.
- TestRenameTableSync is flaky due to KUDU-1353.

master_replication-itest.cc
- TestSysTablesReplication had a weird TODO that didn't make sense. It was
  added here: http://gerrit.sjc.cloudera.com:8080/#/c/5483/20..21. The test
  survives 1000 runs on my laptop after that line is removed though.
- TestCycleThroughAllMasters was flaky due to timeouts in Build(). I suspect
  a default RPC timeout that isn't long enough for TSAN/ASAN builds, as it
  is used as the overall deadline for the leader master RPC "fan out".

Change-Id: I1af480f820f7fce922ed9b9712ee4b6376c352a7
---
M src/kudu/client/client.h
M src/kudu/integration-tests/master_failover-itest.cc
M src/kudu/integration-tests/master_replication-itest.cc
3 files changed, 17 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/68/2368/1
-- 
To view, visit http://gerrit.cloudera.org:8080/2368
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1af480f820f7fce922ed9b9712ee4b6376c352a7
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>