You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2017/09/08 01:18:57 UTC
[kudu-CR] log block manager: use move semantics to fill in the block map
Hello Dan Burkert, Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/8008
to review the following change.
Change subject: log_block_manager: use move semantics to fill in the block map
......................................................................
log_block_manager: use move semantics to fill in the block map
This avoids an extra ref-count increment/decrement while loading
the block map during startup. This substantially improves startup
time.
This shaved off another few percent from LBM startup time.
On a real host with 11M blocks:
Before:
I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s
After:
I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s
According to the benchmark:
Before:
I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s
I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s
I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s
I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s
I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s
I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s
I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s
I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s
I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s
I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s
After:
I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s
I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s
I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s
I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s
I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s
I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s
I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s
I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s
I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s
I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s
Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
---
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
2 files changed, 12 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/8008/1
--
To view, visit http://gerrit.cloudera.org:8080/8008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
[kudu-CR] log block manager: use move semantics to fill in the block map
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: log_block_manager: use move semantics to fill in the block map
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: No
[kudu-CR] log block manager: use move semantics to fill in the block map
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Adar Dembo, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8008
to look at the new patch set (#2).
Change subject: log_block_manager: use move semantics to fill in the block map
......................................................................
log_block_manager: use move semantics to fill in the block map
This avoids an extra ref-count increment/decrement while loading
the block map during startup. This substantially improves startup
time.
This shaved off another few percent from LBM startup time.
On a real host with 11M blocks:
Before:
I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s
After:
I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s
According to the benchmark:
Before:
I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s
I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s
I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s
I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s
I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s
I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s
I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s
I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s
I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s
I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s
After:
I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s
I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s
I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s
I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s
I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s
I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s
I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s
I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s
I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s
I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s
Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
---
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
2 files changed, 14 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/8008/2
--
To view, visit http://gerrit.cloudera.org:8080/8008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] log block manager: use move semantics to fill in the block map
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: log_block_manager: use move semantics to fill in the block map
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8008/1/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:
PS1, Line 2210: UntrackedBlockMap::value_type
Nit: change to auto, if you're interested.
--
To view, visit http://gerrit.cloudera.org:8080/8008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes
[kudu-CR] log block manager: use move semantics to fill in the block map
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.
Change subject: log_block_manager: use move semantics to fill in the block map
......................................................................
log_block_manager: use move semantics to fill in the block map
This avoids an extra ref-count increment/decrement while loading
the block map during startup. This substantially improves startup
time.
This shaved off another few percent from LBM startup time.
On a real host with 11M blocks:
Before:
I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s
After:
I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s
According to the benchmark:
Before:
I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s
I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s
I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s
I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s
I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s
I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s
I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s
I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s
I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s
I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s
After:
I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s
I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s
I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s
I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s
I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s
I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s
I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s
I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s
I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s
I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s
Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Reviewed-on: http://gerrit.cloudera.org:8080/8008
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
2 files changed, 14 insertions(+), 6 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/8008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>