You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2017/08/25 22:42:52 UTC
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Andrew Wong has uploaded a new change for review.
http://gerrit.cloudera.org:8080/7838
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
TestCrashBeforeWriteLogSegmentHeader injects server faults during a
write workload before writing the log segment header. In a stressed test
environment, writes may take longer, and workloads may time out before
even communicating with the server. If the workload never successfully
writes to the server for this reason, and never makes it to the
fault-injection-site, the test will fail.
This is resolved by bumping the timeout allowed by the workload,
ensuring there is enough time to get the table locations and insert data
to them.
With the bumped timeout, this passes 500/500 runs in TSAN with
--stress_cpu_threads=8, rather than failing ~75%.
Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
---
M src/kudu/integration-tests/ts_recovery-itest.cc
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/38/7838/1
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has uploaded a new patch set (#2).
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
TestCrashBeforeWriteLogSegmentHeader injects server faults during a
write workload before writing the log segment header. In a stressed test
environment, writes may take longer, and workloads may time out before
even communicating with the server. If the workload never successfully
writes to the server for this reason, and never makes it to the
fault-injection-site, the test will fail.
This is resolved by bumping the timeout allowed by the workload,
ensuring there is enough time to get the table locations and insert data
to them.
With the bumped timeout, this passes 500/500 runs in TSAN with
--stress_cpu_threads=8, rather than failing ~75%.
Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
---
M src/kudu/integration-tests/ts_recovery-itest.cc
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/38/7838/2
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Patch Set 2:
> That's a safe bet :)
>
> I'm don't have a great explanation. I didn't do a git bisect on it,
> but it seems from the dist-test dash that this has been flaky for
> at least a few days.
All right, at least now the test is stable.
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Patch Set 2: Code-Review+2
I bet this patch had a high time:characters ratio!
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change.
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Patch Set 2:
That's a safe bet :)
I'm don't have a great explanation. I didn't do a git bisect on it, but it seems from the dist-test dash that this has been flaky for at least a few days.
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Patch Set 2: Code-Review+2
BTW, do you have any reasonable explanation why this test started be flakier after the recent updates?
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] Fix flakiness in ts recovery-itest TestCrashBeforeWriteLogSegmentHeader
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.
Change subject: Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
......................................................................
Fix flakiness in ts_recovery-itest TestCrashBeforeWriteLogSegmentHeader
TestCrashBeforeWriteLogSegmentHeader injects server faults during a
write workload before writing the log segment header. In a stressed test
environment, writes may take longer, and workloads may time out before
even communicating with the server. If the workload never successfully
writes to the server for this reason, and never makes it to the
fault-injection-site, the test will fail.
This is resolved by bumping the timeout allowed by the workload,
ensuring there is enough time to get the table locations and insert data
to them.
With the bumped timeout, this passes 500/500 runs in TSAN with
--stress_cpu_threads=8, rather than failing ~75%.
Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Reviewed-on: http://gerrit.cloudera.org:8080/7838
Reviewed-by: Todd Lipcon <to...@apache.org>
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/integration-tests/ts_recovery-itest.cc
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Todd Lipcon: Looks good to me, approved
Alexey Serbin: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/7838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I382bb1109ef12c23d6f191b64d97bcaa6ec96d6a
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>