You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2025/06/12 18:45:58 UTC
[kudu-CR] [tablet] less re-allocation of vectors in RowsetTree
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23018
Change subject: [tablet] less re-allocation of vectors in RowsetTree
......................................................................
[tablet] less re-allocation of vectors in RowsetTree
Since row check presence is in the hot path for high intensity ingestion
and update workloads, it makes sense to minimize the number of memory
reallocations to help with lock contention in tcmalloc. I found it's
quite trivial to do so for Tablet::CaptureConsistentIterators() and
RowSetTree::FindRowSets{WithKeyInRange,IntersectingInterval}(), and this
is addressed here. For more context, in the field we saw extreme
situations when tens (or even 100+) rowset iterators were present
for a scanner per reported metrics, so several reallocations might
be done in RowSetTree::FindRowSets...() methods when growing std::vector
with the standard capacity-doubling strategy.
I also updated rowset_tree-test.cc to use std::stable_sort() instead
of std::sort() to match how it's done in Tablet::BulkCheckPresence().
With that, the performance numbers on bulk key presence checks reported
by TestRowSetTreePerformance scenarios in rowset_tree.cc should be
closer to what it takes in Tablet::BulkCheckPresence().
I verified that the performance with this patch didn't degrade at least
as per the numbers reported by benchmarking scenarios in rowset_tree.cc.
This changelist doesn't contains any functional modifications.
Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
---
M src/kudu/tablet/rowset_tree-test.cc
M src/kudu/tablet/rowset_tree.cc
M src/kudu/tablet/rowset_tree.h
M src/kudu/tablet/tablet.cc
4 files changed, 52 insertions(+), 38 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/23018/1
--
To view, visit http://gerrit.cloudera.org:8080/23018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Gerrit-Change-Number: 23018
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
[kudu-CR] [tablet] less re-allocation of vectors in RowsetTree
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23018 )
Change subject: [tablet] less re-allocation of vectors in RowsetTree
......................................................................
Patch Set 1: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/1446/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/23018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Gerrit-Change-Number: 23018
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jun 2025 23:24:20 +0000
Gerrit-HasComments: No
[kudu-CR] [tablet] less re-allocation of vectors in RowsetTree
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23018 )
Change subject: [tablet] less re-allocation of vectors in RowsetTree
......................................................................
Patch Set 1:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/1446/
--
To view, visit http://gerrit.cloudera.org:8080/23018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Gerrit-Change-Number: 23018
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 12 Jun 2025 18:46:04 +0000
Gerrit-HasComments: No
[kudu-CR] [tablet] less re-allocation of vectors in RowsetTree
Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/23018 )
Change subject: [tablet] less re-allocation of vectors in RowsetTree
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/23018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Gerrit-Change-Number: 23018
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Jun 2025 16:48:27 +0000
Gerrit-HasComments: No
[kudu-CR] [tablet] less re-allocation of vectors in RowsetTree
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23018 )
Change subject: [tablet] less re-allocation of vectors in RowsetTree
......................................................................
[tablet] less re-allocation of vectors in RowsetTree
Since row check presence is in the hot path for high intensity ingestion
and update workloads, it makes sense to minimize the number of memory
reallocations to help with lock contention in tcmalloc. I found it's
quite trivial to do so for Tablet::CaptureConsistentIterators() and
RowSetTree::FindRowSets{WithKeyInRange,IntersectingInterval}(), and this
is addressed here. For more context, in the field we saw extreme
situations when tens (or even 100+) rowset iterators were present
for a scanner per reported metrics, so several reallocations might
be done in RowSetTree::FindRowSets...() methods when growing std::vector
with the standard capacity-doubling strategy.
I also updated rowset_tree-test.cc to use std::stable_sort() instead
of std::sort() to match how it's done in Tablet::BulkCheckPresence().
With that, the performance numbers on bulk key presence checks reported
by TestRowSetTreePerformance scenarios in rowset_tree.cc should be
closer to what it takes in Tablet::BulkCheckPresence().
I verified that the performance with this patch didn't degrade at least
as per the numbers reported by benchmarking scenarios in rowset_tree.cc.
This changelist doesn't contains any functional modifications.
Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Reviewed-on: http://gerrit.cloudera.org:8080/23018
Tested-by: Kudu Jenkins
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
---
M src/kudu/tablet/rowset_tree-test.cc
M src/kudu/tablet/rowset_tree.cc
M src/kudu/tablet/rowset_tree.h
M src/kudu/tablet/tablet.cc
4 files changed, 52 insertions(+), 38 deletions(-)
Approvals:
Kudu Jenkins: Verified
Abhishek Chennaka: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/23018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia5ffa0cce6b7a94567a7316c4909c7bbcfa1439c
Gerrit-Change-Number: 23018
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>