You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@kudu.apache.org by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org> on 2016/10/06 04:50:16 UTC

[kudu-CR] [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest

David Ribeiro Alves has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4644

Change subject: [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest
......................................................................

[flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest

In this test we have two writers, one that is performs a series normal
writes and one that stubbornly tries the same request. The latter is supposed
to observe stale requests, the former isn't but is observing them anyway.
This is happening the result ttl is very low and the RPC would sometimes
fail/retry naturally and find that the response had been gc'd.

This just increases the result ttl to 50 msecs, which should still leave
more than enough time for the stubborn writer to see stale requests.

Change-Id: I1257ddaf02dfca05829edc904b2f4ff406b933ca
---
M src/kudu/rpc/exactly_once_rpc-test.cc
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/44/4644/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4644
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1257ddaf02dfca05829edc904b2f4ff406b933ca
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>

[kudu-CR] [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.

David Ribeiro Alves has abandoned this change. ( http://gerrit.cloudera.org:8080/4644 )

Change subject: [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest
......................................................................


Abandoned

abandoning this, not the root cause of the problems
-- 
To view, visit http://gerrit.cloudera.org:8080/4644
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I1257ddaf02dfca05829edc904b2f4ff406b933ca
Gerrit-Change-Number: 4644
Gerrit-PatchSet: 1
Gerrit-Owner: David Ribeiro Alves <da...@gmail.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.

Todd Lipcon has posted comments on this change.

Change subject: [flaky tests] Increase the result ttl for ExactlyOnceRpcTest.TestExactlyOnceSemanticsGarbageCollectionStressTest
......................................................................


Patch Set 1:

I don't think this is exactly the problem. Instead, I think the issue is the following interleaving:

- the LongWritesThread generates sequence number 1 (from the result tracker)
- the StubbornWriteThreads generates a sequence number 2 (from the result tracker)
- the StubbornWritesThread manually sets first_incomplete_seq_no in its RPC to 2 (its own sequence number)
- the StubbornWritesThread sends an RPC, which sets the server side "stale" threshold to 2
- the LongWritesThread continues running and sends sequence number 1, which is now marked as stale, and fails.

If you add a usleep(5000) at the top of CalculatureServiceRpc::Try() you'll trigger this interleaving regularly even with your patch.

I think the real fix here is that the AddRequestId() call should be using the request_tracker's view of the first_incomplete_seq_no rather than setting it equal to the request's sequence number.

-- 
To view, visit http://gerrit.cloudera.org:8080/4644
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1257ddaf02dfca05829edc904b2f4ff406b933ca
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No