You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Will Berkeley (Code Review)" <ge...@cloudera.org> on 2019/06/03 06:31:26 UTC
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Will Berkeley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13496
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
[java] Fix ordering bug in AsyncKuduSession
AsyncKuduSession.testBatchErrorCauseSessionStuck occasionally failed
because AsyncKuduSession#hasPendingOperations returned true even though
the test joined on the deferred results of its two inserts. I tracked
this down to a small mis-ordering of lines where the AsyncKuduSession
fires a flush notification (from a Netty worker thread) before it
actually swaps the just-flushed buffer back to the inactive list, so
there was a small window of time when the client thread could observe
hasPendingOperations() as true even though there were no pending
operations.
I ran the test 100 times in dist-test with this fix and saw no failures.
I didn't run the test pre-fix through dist-test but the flaky dashboard
indicated it was 2-3% flaky.
Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/13496/1
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13496 )
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@306
PS1, Line 306: * Returns a buffer to the inactive queue after flushing.
nit: Can you update this comment since the order changed? Maybe:
Returns a buffer to the inactive queue and flushes the buffer.
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 03 Jun 2019 16:14:14 +0000
Gerrit-HasComments: Yes
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13496 )
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@306
PS1, Line 306: * Returns a buffer to the inactive queue after flushing.
> nit: Can you update this comment since the order changed? Maybe:
The buffer is already flushed at this point. This function resets state post-flush.
I'm thinking about rewriting some of the internals of AsyncKuduSession to make buffer management easier to understand.
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 03 Jun 2019 16:47:42 +0000
Gerrit-HasComments: Yes
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13496 )
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/13496/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@306
PS1, Line 306: * Returns a buffer to the inactive queue after flushing.
> The buffer is already flushed at this point. This function resets state pos
Sounds good. We can commit this as is and do that in follow up work.
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 03 Jun 2019 16:50:29 +0000
Gerrit-HasComments: Yes
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13496 )
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
[java] Fix ordering bug in AsyncKuduSession
AsyncKuduSession.testBatchErrorCauseSessionStuck occasionally failed
because AsyncKuduSession#hasPendingOperations returned true even though
the test joined on the deferred results of its two inserts. I tracked
this down to a small mis-ordering of lines where the AsyncKuduSession
fires a flush notification (from a Netty worker thread) before it
actually swaps the just-flushed buffer back to the inactive list, so
there was a small window of time when the client thread could observe
hasPendingOperations() as true even though there were no pending
operations.
I ran the test 100 times in dist-test with this fix and saw no failures.
I didn't run the test pre-fix through dist-test but the flaky dashboard
indicated it was 2-3% flaky.
Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Reviewed-on: http://gerrit.cloudera.org:8080/13496
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Kudu Jenkins: Verified
Grant Henke: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [java] Fix ordering bug in AsyncKuduSession
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13496 )
Change subject: [java] Fix ordering bug in AsyncKuduSession
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13496
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I498636aeb0de2673c15b90260e6acf56de1ead06
Gerrit-Change-Number: 13496
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 03 Jun 2019 16:52:03 +0000
Gerrit-HasComments: No