You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2021/12/11 07:15:52 UTC
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18090
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
[consensus] reduce critical section in LogCache::ReadOps()
This patch reduces the section protected by the lock in
LogCache::ReadOps(), moving out of the critical section several
operations which do not need such a protection.
I didn't find an existing performance test which would exercise the
updated code path, but reducing the time spent under a lock looks like
an improvement.
Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
---
M src/kudu/consensus/log_cache.cc
1 file changed, 13 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/18090/1
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18090/1//COMMIT_MSG
Commit Message:
PS1:
Heh, it seems this patch breaks something: ConsensusQueueTest.TestGetPagedMessages failed. I'll take a closer look next week.
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 11 Dec 2021 07:23:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18090
to look at the new patch set (#2).
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
[consensus] reduce critical section in LogCache::ReadOps()
This patch reduces the section protected by the lock in
LogCache::ReadOps(), moving out of the critical section several
operations which do not need such a protection.
I didn't find an existing performance test which would exercise the
updated code path, but reducing the time spent under a lock looks like
an improvement.
Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
---
M src/kudu/consensus/consensus_queue-test.cc
M src/kudu/consensus/log_cache.cc
2 files changed, 20 insertions(+), 22 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/18090/2
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc
File src/kudu/consensus/log_cache.cc:
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc@348
PS1, Line 348: if (remaining_space < 0) {
> Yup, that's the idea: once we do subtraction of the total message size belo
It seems on of the tests relies on this exact logic, so I returned it back. Essentially, this does pre-check on whether we are about to get over the limit if adding an extra element (but we have to provide at least one element in the output).
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 15 Dec 2021 23:12:17 +0000
Gerrit-HasComments: Yes
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Andrew Wong from this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Removed reviewer Andrew Wong.
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 15 Dec 2021 23:39:55 +0000
Gerrit-HasComments: No
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc
File src/kudu/consensus/log_cache.cc:
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc@348
PS1, Line 348: if (remaining_space < 0) {
Don't we have to do the TotalByteSizeForMessage() subtraction before this? Otherwise, isn't it possible 'messages' contains more bytes than we have capacity for?
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 14 Dec 2021 23:17:20 +0000
Gerrit-HasComments: Yes
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
Patch Set 1:
(1 comment)
Thank you for taking a look!
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc
File src/kudu/consensus/log_cache.cc:
http://gerrit.cloudera.org:8080/#/c/18090/1/src/kudu/consensus/log_cache.cc@348
PS1, Line 348: if (remaining_space < 0) {
> Don't we have to do the TotalByteSizeForMessage() subtraction before this?
Yup, that's the idea: once we do subtraction of the total message size below, we don't need to check for messages->empty() anymore given that the outer 'while' cycle has (remaining_space > 0) as the part of the criterion.
And it might be possible to get more bytes there if that's the only message, yes (and it should be so -- otherwise this code could not be able to fetch even a single message and it would stall).
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 15 Dec 2021 00:14:15 +0000
Gerrit-HasComments: Yes
[kudu-CR] [consensus] reduce critical section in LogCache::ReadOps()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18090 )
Change subject: [consensus] reduce critical section in LogCache::ReadOps()
......................................................................
[consensus] reduce critical section in LogCache::ReadOps()
This patch reduces the section protected by the lock in
LogCache::ReadOps(), moving out of the critical section several
operations which do not need such a protection.
I didn't find an existing performance test which would exercise the
updated code path, but reducing the time spent under a lock looks like
an improvement.
Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Reviewed-on: http://gerrit.cloudera.org:8080/18090
Reviewed-by: Andrew Wong <aw...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/consensus/consensus_queue-test.cc
M src/kudu/consensus/log_cache.cc
2 files changed, 20 insertions(+), 22 deletions(-)
Approvals:
Andrew Wong: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia7de32eb8adde242c8047720e062535cbdaf3e39
Gerrit-Change-Number: 18090
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)