You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2019/12/04 06:29:22 UTC

[kudu-CR] [test] fix flakiness in kudu-admin-test scenario

Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14827


Change subject: [test] fix flakiness in kudu-admin-test scenario
......................................................................

[test] fix flakiness in kudu-admin-test scenario

I saw AdminCliTest.TestSimultaneousLeaderTransferAndAbruptStepdown
failing with timed-out error trying to write data to the test table:

  http://dist-test.cloudera.org/job?job_id=aserbin.1575418627.126650

It seems the frequency of the leader change/step-down requests might be
safely decreased to make this test more stable.  The test already have
the frequency of the leader change requests clamped down for ASAN
builds compared with the RELEASE/DEBUG/TSAN case.

Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
---
M src/kudu/tools/kudu-admin-test.cc
1 file changed, 1 insertion(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/14827/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14827/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14827/1//COMMIT_MSG@7
PS1, Line 7: [test] fix flakiness in kudu-admin-test scenario
> This is KUDU-2610, right? Could you review that ticket and see if it's abou
Indeed: that's KUDU-2610.  Thanks!



-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 05 Dec 2019 04:56:15 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.

Change subject: [test] fix flakiness in kudu-admin-test scenario
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................

[test] KUDU-2610 fix flakiness in kudu-admin-test scenario

I saw AdminCliTest.TestSimultaneousLeaderTransferAndAbruptStepdown
failing with timed-out error trying to write data to the test table:

  http://dist-test.cloudera.org/job?job_id=aserbin.1575418627.126650

A log from another occurrence of this issue is attached to
https://issues.apache.org/jira/browse/KUDU-2610

It seems the frequency of the leader change/step-down requests might be
safely decreased to make this test more stable.  The test already have
the frequency of the leader change requests clamped down for ASAN
builds compared with the RELEASE/DEBUG/TSAN case.

Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Reviewed-on: http://gerrit.cloudera.org:8080/14827
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/tools/kudu-admin-test.cc
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Adar Dembo: Looks good to me, approved
  Kudu Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14827/2/src/kudu/tools/kudu-admin-test.cc
File src/kudu/tools/kudu-admin-test.cc:

PS2: 
> Note that the bug report was filed for an ASAN failure, suggesting that per
Indeed.

I increased the period for leader elections in case of ASAN up to 6 seconds.  Hopefully, that should be good enough.  At least, running the test several thousand times didn't expose any flakiness with new settings.



-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 05 Dec 2019 06:22:37 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 05 Dec 2019 05:39:18 +0000
Gerrit-HasComments: No

[kudu-CR] [test] fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 1: Verified+1

Unrelated test failure in org.apache.kudu.client.TestAsyncKuduSession


-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 04 Dec 2019 18:03:42 +0000
Gerrit-HasComments: No

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14827

to look at the new patch set (#3).

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................

[test] KUDU-2610 fix flakiness in kudu-admin-test scenario

I saw AdminCliTest.TestSimultaneousLeaderTransferAndAbruptStepdown
failing with timed-out error trying to write data to the test table:

  http://dist-test.cloudera.org/job?job_id=aserbin.1575418627.126650

A log from another occurrence of this issue is attached to
https://issues.apache.org/jira/browse/KUDU-2610

It seems the frequency of the leader change/step-down requests might be
safely decreased to make this test more stable.  The test already have
the frequency of the leader change requests clamped down for ASAN
builds compared with the RELEASE/DEBUG/TSAN case.

Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
---
M src/kudu/tools/kudu-admin-test.cc
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/14827/3
-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 2: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14827/2/src/kudu/tools/kudu-admin-test.cc
File src/kudu/tools/kudu-admin-test.cc:

PS2: 
Note that the bug report was filed for an ASAN failure, suggesting that perhaps the ASAN-specific value is still too low.



-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 05 Dec 2019 05:10:18 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] fix flakiness in kudu-admin-test scenario

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14827 )

Change subject: [test] fix flakiness in kudu-admin-test scenario
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14827/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14827/1//COMMIT_MSG@7
PS1, Line 7: [test] fix flakiness in kudu-admin-test scenario
This is KUDU-2610, right? Could you review that ticket and see if it's about the same problem?



-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 04 Dec 2019 23:32:56 +0000
Gerrit-HasComments: Yes

[kudu-CR] [test] KUDU-2610 fix flakiness in kudu-admin-test scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14827

to look at the new patch set (#2).

Change subject: [test] KUDU-2610 fix flakiness in kudu-admin-test scenario
......................................................................

[test] KUDU-2610 fix flakiness in kudu-admin-test scenario

I saw AdminCliTest.TestSimultaneousLeaderTransferAndAbruptStepdown
failing with timed-out error trying to write data to the test table:

  http://dist-test.cloudera.org/job?job_id=aserbin.1575418627.126650

A log from another occurrence of this issue is attached to
https://issues.apache.org/jira/browse/KUDU-2610

It seems the frequency of the leader change/step-down requests might be
safely decreased to make this test more stable.  The test already have
the frequency of the leader change requests clamped down for ASAN
builds compared with the RELEASE/DEBUG/TSAN case.

Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
---
M src/kudu/tools/kudu-admin-test.cc
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/14827/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14827
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I98e792783efa2909d10174f84ddd785f5a968046
Gerrit-Change-Number: 14827
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)