You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2020/07/09 23:06:03 UTC
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Hello Alexey Serbin,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/16162
to review the following change.
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
KUDU-636. Use Arena for EncodedKeys
This updates EncodedKeyBuilder, RowSetKeyProbe, and EncodedKey to always
allocate from an Arena instead of from the heap. This reduces allocator
contention on the write path significantly and improves memory locality.
I measured by running a tserver under 'perf stat' while using perf loadgen to
insert 8M rows total using 8 client threads. The CPU time on the tserver was
reduced by about 20%.
Before:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
269853.10 msec task-clock # 6.862 CPUs utilized
293066 context-switches # 0.001 M/sec
44541 cpu-migrations # 0.165 K/sec
2846435 page-faults # 0.011 M/sec
1110190206891 cycles # 4.114 GHz (83.33%)
201895623339 stalled-cycles-frontend # 18.19% frontend cycles idle (83.33%)
137095475307 stalled-cycles-backend # 12.35% backend cycles idle (83.32%)
894201276095 instructions # 0.81 insn per cycle
# 0.23 stalled cycles per insn (83.33%)
159095264762 branches # 589.562 M/sec (83.35%)
639216492 branch-misses # 0.40% of all branches (83.35%)
255.178068000 seconds user
14.913394000 seconds sys
After:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
227730.62 msec task-clock # 6.212 CPUs utilized
263824 context-switches # 0.001 M/sec
45470 cpu-migrations # 0.200 K/sec
3165436 page-faults # 0.014 M/sec
931840588715 cycles # 4.092 GHz (83.25%)
183214671009 stalled-cycles-frontend # 19.66% frontend cycles idle (83.40%)
111864991317 stalled-cycles-backend # 12.00% backend cycles idle (83.35%)
832636863971 instructions # 0.89 insn per cycle
# 0.22 stalled cycles per insn (83.40%)
148228107120 branches # 650.892 M/sec (83.24%)
563344647 branch-misses # 0.38% of all branches (83.35%)
211.361472000 seconds user
16.635265000 seconds sys
Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
---
M src/kudu/cfile/cfile-test.cc
M src/kudu/client/scan_configuration.cc
M src/kudu/client/write_op.cc
M src/kudu/client/write_op.h
M src/kudu/common/encoded_key-test.cc
M src/kudu/common/encoded_key.cc
M src/kudu/common/encoded_key.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/scan_spec.cc
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/row_op.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet-test.cc
M src/kudu/tablet/tablet.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_server_authorization-test.cc
M src/kudu/tserver/tablet_service.cc
22 files changed, 219 insertions(+), 240 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/16162/1
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/client/write_op.cc
File src/kudu/client/write_op.cc:
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/client/write_op.cc@a61
PS2, Line 61:
:
:
:
:
:
:
:
:
:
:
> Interesting, at what point it stopped being used? :)
looks like in 3721be1723f2402e5fe75898349a03d348ace7c1
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc
File src/kudu/common/encoded_key.cc:
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc@56
PS2, Line 56: Status EncodedKey::DecodeEncodedString(const Schema& schema, Arena* arena, const Slice& encoded,
: EncodedKey** result) {
> A nit for here and elsewhere: it seems there was a tendency to have one par
I was trying to use clang-format-diff with the Google style, which ends up doing argument/parameter binpacking by default. I overrode those options and it went back to something closer to the one-per-line style in these cases.
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 10 Jul 2020 16:16:02 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16162
to look at the new patch set (#2).
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
KUDU-636. Use Arena for EncodedKeys
This updates EncodedKeyBuilder, RowSetKeyProbe, and EncodedKey to always
allocate from an Arena instead of from the heap. This reduces allocator
contention on the write path significantly and improves memory locality.
I measured by running a tserver under 'perf stat' while using perf loadgen to
insert 8M rows total using 8 client threads. The CPU time on the tserver was
reduced by about 20%.
Before:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
269853.10 msec task-clock # 6.862 CPUs utilized
293066 context-switches # 0.001 M/sec
44541 cpu-migrations # 0.165 K/sec
2846435 page-faults # 0.011 M/sec
1110190206891 cycles # 4.114 GHz (83.33%)
201895623339 stalled-cycles-frontend # 18.19% frontend cycles idle (83.33%)
137095475307 stalled-cycles-backend # 12.35% backend cycles idle (83.32%)
894201276095 instructions # 0.81 insn per cycle
# 0.23 stalled cycles per insn (83.33%)
159095264762 branches # 589.562 M/sec (83.35%)
639216492 branch-misses # 0.40% of all branches (83.35%)
255.178068000 seconds user
14.913394000 seconds sys
After:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
227730.62 msec task-clock # 6.212 CPUs utilized
263824 context-switches # 0.001 M/sec
45470 cpu-migrations # 0.200 K/sec
3165436 page-faults # 0.014 M/sec
931840588715 cycles # 4.092 GHz (83.25%)
183214671009 stalled-cycles-frontend # 19.66% frontend cycles idle (83.40%)
111864991317 stalled-cycles-backend # 12.00% backend cycles idle (83.35%)
832636863971 instructions # 0.89 insn per cycle
# 0.22 stalled cycles per insn (83.40%)
148228107120 branches # 650.892 M/sec (83.24%)
563344647 branch-misses # 0.38% of all branches (83.35%)
211.361472000 seconds user
16.635265000 seconds sys
Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
---
M src/kudu/cfile/cfile-test.cc
M src/kudu/client/scan_configuration.cc
M src/kudu/client/write_op.cc
M src/kudu/client/write_op.h
M src/kudu/common/encoded_key-test.cc
M src/kudu/common/encoded_key.cc
M src/kudu/common/encoded_key.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/scan_spec.cc
M src/kudu/common/scan_spec.h
M src/kudu/tablet/all_types-scan-correctness-test.cc
M src/kudu/tablet/cfile_set-test.cc
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/row_op.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet-decoder-eval-test.cc
M src/kudu/tablet/tablet-pushdown-test.cc
M src/kudu/tablet/tablet-test.cc
M src/kudu/tablet/tablet.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/scanners.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_server_authorization-test.cc
M src/kudu/tserver/tablet_service.cc
28 files changed, 266 insertions(+), 337 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/16162/2
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Patch Set 4: Verified+1
Test failure looks unrelated
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 10 Jul 2020 18:55:16 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
KUDU-636. Use Arena for EncodedKeys
This updates EncodedKeyBuilder, RowSetKeyProbe, and EncodedKey to always
allocate from an Arena instead of from the heap. This reduces allocator
contention on the write path significantly and improves memory locality.
I measured by running a tserver under 'perf stat' while using perf loadgen to
insert 80M rows total using 8 client threads. The CPU time on the tserver was
reduced by about 20%.
Before:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
269853.10 msec task-clock # 6.862 CPUs utilized
293066 context-switches # 0.001 M/sec
44541 cpu-migrations # 0.165 K/sec
2846435 page-faults # 0.011 M/sec
1110190206891 cycles # 4.114 GHz (83.33%)
201895623339 stalled-cycles-frontend # 18.19% frontend cycles idle (83.33%)
137095475307 stalled-cycles-backend # 12.35% backend cycles idle (83.32%)
894201276095 instructions # 0.81 insn per cycle
# 0.23 stalled cycles per insn (83.33%)
159095264762 branches # 589.562 M/sec (83.35%)
639216492 branch-misses # 0.40% of all branches (83.35%)
255.178068000 seconds user
14.913394000 seconds sys
After:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
227730.62 msec task-clock # 6.212 CPUs utilized
263824 context-switches # 0.001 M/sec
45470 cpu-migrations # 0.200 K/sec
3165436 page-faults # 0.014 M/sec
931840588715 cycles # 4.092 GHz (83.25%)
183214671009 stalled-cycles-frontend # 19.66% frontend cycles idle (83.40%)
111864991317 stalled-cycles-backend # 12.00% backend cycles idle (83.35%)
832636863971 instructions # 0.89 insn per cycle
# 0.22 stalled cycles per insn (83.40%)
148228107120 branches # 650.892 M/sec (83.24%)
563344647 branch-misses # 0.38% of all branches (83.35%)
211.361472000 seconds user
16.635265000 seconds sys
Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Reviewed-on: http://gerrit.cloudera.org:8080/16162
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Todd Lipcon <to...@apache.org>
---
M src/kudu/cfile/cfile-test.cc
M src/kudu/client/scan_configuration.cc
M src/kudu/client/write_op.cc
M src/kudu/client/write_op.h
M src/kudu/common/encoded_key-test.cc
M src/kudu/common/encoded_key.cc
M src/kudu/common/encoded_key.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/scan_spec.cc
M src/kudu/common/scan_spec.h
M src/kudu/tablet/all_types-scan-correctness-test.cc
M src/kudu/tablet/cfile_set-test.cc
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/row_op.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet-decoder-eval-test.cc
M src/kudu/tablet/tablet-pushdown-test.cc
M src/kudu/tablet/tablet-test.cc
M src/kudu/tablet/tablet.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/scanners.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_server_authorization-test.cc
M src/kudu/tserver/tablet_service.cc
28 files changed, 265 insertions(+), 330 deletions(-)
Approvals:
Alexey Serbin: Looks good to me, approved
Todd Lipcon: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc
File src/kudu/common/encoded_key.cc:
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc@56
PS2, Line 56: Status EncodedKey::DecodeEncodedString(const Schema& schema,
: Arena* arena,
> I was trying to use clang-format-diff with the Google style, which ends up
Thank you for addressing that!
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 10 Jul 2020 16:50:10 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has removed a vote on this change.
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Patch Set 2: Code-Review+2
(2 comments)
Looks good overall, just one style nit.
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/client/write_op.cc
File src/kudu/client/write_op.cc:
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/client/write_op.cc@a61
PS2, Line 61:
:
:
:
:
:
:
:
:
:
:
Interesting, at what point it stopped being used? :)
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc
File src/kudu/common/encoded_key.cc:
http://gerrit.cloudera.org:8080/#/c/16162/2/src/kudu/common/encoded_key.cc@56
PS2, Line 56: Status EncodedKey::DecodeEncodedString(const Schema& schema, Arena* arena, const Slice& encoded,
: EncodedKey** result) {
A nit for here and elsewhere: it seems there was a tendency to have one parameter per line, which is better from the readability perspective. I guess the idea was that forming one-per-line or all-at-same-line lists are easier to read.
Do you mind keeping that formatting in the modified files?
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Fri, 10 Jul 2020 06:41:29 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16162
to look at the new patch set (#4).
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
KUDU-636. Use Arena for EncodedKeys
This updates EncodedKeyBuilder, RowSetKeyProbe, and EncodedKey to always
allocate from an Arena instead of from the heap. This reduces allocator
contention on the write path significantly and improves memory locality.
I measured by running a tserver under 'perf stat' while using perf loadgen to
insert 80M rows total using 8 client threads. The CPU time on the tserver was
reduced by about 20%.
Before:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
269853.10 msec task-clock # 6.862 CPUs utilized
293066 context-switches # 0.001 M/sec
44541 cpu-migrations # 0.165 K/sec
2846435 page-faults # 0.011 M/sec
1110190206891 cycles # 4.114 GHz (83.33%)
201895623339 stalled-cycles-frontend # 18.19% frontend cycles idle (83.33%)
137095475307 stalled-cycles-backend # 12.35% backend cycles idle (83.32%)
894201276095 instructions # 0.81 insn per cycle
# 0.23 stalled cycles per insn (83.33%)
159095264762 branches # 589.562 M/sec (83.35%)
639216492 branch-misses # 0.40% of all branches (83.35%)
255.178068000 seconds user
14.913394000 seconds sys
After:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
227730.62 msec task-clock # 6.212 CPUs utilized
263824 context-switches # 0.001 M/sec
45470 cpu-migrations # 0.200 K/sec
3165436 page-faults # 0.014 M/sec
931840588715 cycles # 4.092 GHz (83.25%)
183214671009 stalled-cycles-frontend # 19.66% frontend cycles idle (83.40%)
111864991317 stalled-cycles-backend # 12.00% backend cycles idle (83.35%)
832636863971 instructions # 0.89 insn per cycle
# 0.22 stalled cycles per insn (83.40%)
148228107120 branches # 650.892 M/sec (83.24%)
563344647 branch-misses # 0.38% of all branches (83.35%)
211.361472000 seconds user
16.635265000 seconds sys
Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
---
M src/kudu/cfile/cfile-test.cc
M src/kudu/client/scan_configuration.cc
M src/kudu/client/write_op.cc
M src/kudu/client/write_op.h
M src/kudu/common/encoded_key-test.cc
M src/kudu/common/encoded_key.cc
M src/kudu/common/encoded_key.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/scan_spec.cc
M src/kudu/common/scan_spec.h
M src/kudu/tablet/all_types-scan-correctness-test.cc
M src/kudu/tablet/cfile_set-test.cc
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/row_op.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet-decoder-eval-test.cc
M src/kudu/tablet/tablet-pushdown-test.cc
M src/kudu/tablet/tablet-test.cc
M src/kudu/tablet/tablet.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/scanners.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_server_authorization-test.cc
M src/kudu/tserver/tablet_service.cc
28 files changed, 265 insertions(+), 330 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/16162/4
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16162
to look at the new patch set (#3).
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
KUDU-636. Use Arena for EncodedKeys
This updates EncodedKeyBuilder, RowSetKeyProbe, and EncodedKey to always
allocate from an Arena instead of from the heap. This reduces allocator
contention on the write path significantly and improves memory locality.
I measured by running a tserver under 'perf stat' while using perf loadgen to
insert 8M rows total using 8 client threads. The CPU time on the tserver was
reduced by about 20%.
Before:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
269853.10 msec task-clock # 6.862 CPUs utilized
293066 context-switches # 0.001 M/sec
44541 cpu-migrations # 0.165 K/sec
2846435 page-faults # 0.011 M/sec
1110190206891 cycles # 4.114 GHz (83.33%)
201895623339 stalled-cycles-frontend # 18.19% frontend cycles idle (83.33%)
137095475307 stalled-cycles-backend # 12.35% backend cycles idle (83.32%)
894201276095 instructions # 0.81 insn per cycle
# 0.23 stalled cycles per insn (83.33%)
159095264762 branches # 589.562 M/sec (83.35%)
639216492 branch-misses # 0.40% of all branches (83.35%)
255.178068000 seconds user
14.913394000 seconds sys
After:
Performance counter stats for './build/latest/bin/kudu tserver run -fs-wal-dir /tmp/ts':
227730.62 msec task-clock # 6.212 CPUs utilized
263824 context-switches # 0.001 M/sec
45470 cpu-migrations # 0.200 K/sec
3165436 page-faults # 0.014 M/sec
931840588715 cycles # 4.092 GHz (83.25%)
183214671009 stalled-cycles-frontend # 19.66% frontend cycles idle (83.40%)
111864991317 stalled-cycles-backend # 12.00% backend cycles idle (83.35%)
832636863971 instructions # 0.89 insn per cycle
# 0.22 stalled cycles per insn (83.40%)
148228107120 branches # 650.892 M/sec (83.24%)
563344647 branch-misses # 0.38% of all branches (83.35%)
211.361472000 seconds user
16.635265000 seconds sys
Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
---
M src/kudu/cfile/cfile-test.cc
M src/kudu/client/scan_configuration.cc
M src/kudu/client/write_op.cc
M src/kudu/client/write_op.h
M src/kudu/common/encoded_key-test.cc
M src/kudu/common/encoded_key.cc
M src/kudu/common/encoded_key.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/scan_spec.cc
M src/kudu/common/scan_spec.h
M src/kudu/tablet/all_types-scan-correctness-test.cc
M src/kudu/tablet/cfile_set-test.cc
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset-test-base.h
M src/kudu/tablet/diskrowset-test.cc
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/row_op.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet-decoder-eval-test.cc
M src/kudu/tablet/tablet-pushdown-test.cc
M src/kudu/tablet/tablet-test.cc
M src/kudu/tablet/tablet.cc
M src/kudu/tools/tool_action_fs.cc
M src/kudu/tserver/scanners.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_server_authorization-test.cc
M src/kudu/tserver/tablet_service.cc
28 files changed, 265 insertions(+), 330 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/16162/3
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] KUDU-636. Use Arena for EncodedKeys
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16162 )
Change subject: KUDU-636. Use Arena for EncodedKeys
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16162
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib46d0e2c31e03a7f319ceb0bf742e08ff74d7683
Gerrit-Change-Number: 16162
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 10 Jul 2020 17:04:42 +0000
Gerrit-HasComments: No