You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2017/09/08 01:18:57 UTC
[kudu-CR] WIP: pb util: avoid repeated stat() calls reading files
Hello Dan Burkert, Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/8010
to review the following change.
Change subject: WIP: pb_util: avoid repeated stat() calls reading files
......................................................................
WIP: pb_util: avoid repeated stat() calls reading files
This reduces the number of fstat syscalls while loading a host with 11M blocks
from 29.3M to 147K.
Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
---
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
2 files changed, 29 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/8010/1
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
[kudu-CR] pb util: avoid repeated stat() calls reading files
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/8010 )
Change subject: pb_util: avoid repeated stat() calls reading files
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-Change-Number: 8010
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Tue, 10 Oct 2017 18:07:21 +0000
Gerrit-HasComments: No
[kudu-CR] pb util: avoid repeated stat() calls reading files
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8010
to look at the new patch set (#4).
Change subject: pb_util: avoid repeated stat() calls reading files
......................................................................
pb_util: avoid repeated stat() calls reading files
This reduces the number of fstat syscalls while loading a host with 11M blocks
from 29.3M to 147K.
Note that this also changes Env to return EndOfFile when reading from
disk rather than IOError as it used to.
Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
---
M src/kudu/util/env-test.cc
M src/kudu/util/env_posix.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
5 files changed, 62 insertions(+), 22 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/8010/4
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-Change-Number: 8010
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] pb util: avoid repeated stat() calls reading files
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/8010 )
Change subject: pb_util: avoid repeated stat() calls reading files
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-Change-Number: 8010
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Tue, 10 Oct 2017 00:03:53 +0000
Gerrit-HasComments: No
[kudu-CR] WIP: pb util: avoid repeated stat() calls reading files
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8010
to look at the new patch set (#2).
Change subject: WIP: pb_util: avoid repeated stat() calls reading files
......................................................................
WIP: pb_util: avoid repeated stat() calls reading files
This reduces the number of fstat syscalls while loading a host with 11M blocks
from 29.3M to 147K.
(just rebased, tests still broken)
Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
---
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
2 files changed, 29 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/8010/2
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] pb util: avoid repeated stat() calls reading files
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8010 )
Change subject: pb_util: avoid repeated stat() calls reading files
......................................................................
pb_util: avoid repeated stat() calls reading files
This reduces the number of fstat syscalls while loading a host with 11M blocks
from 29.3M to 147K.
Note that this also changes Env to return EndOfFile when reading from
disk rather than IOError as it used to.
Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Reviewed-on: http://gerrit.cloudera.org:8080/8010
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/util/env-test.cc
M src/kudu/util/env_posix.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
5 files changed, 62 insertions(+), 22 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-Change-Number: 8010
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] WIP: pb util: avoid repeated stat() calls reading files
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: WIP: pb_util: avoid repeated stat() calls reading files
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/8010/1/src/kudu/util/pb_util.cc
File src/kudu/util/pb_util.cc:
PS1, Line 260: if (!*cached_file_size) {
: RETURN_NOT_OK(reader->Size(&file_size));
: *cached_file_size = file_size;
: } else {
: file_size = cached_file_size->get();
: }
Nit: invert for clarity?
Line 355: RETURN_NOT_OK(reader->Size(&file_size));
What about this call?
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes
[kudu-CR] pb util: avoid repeated stat() calls reading files
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8010
to look at the new patch set (#3).
Change subject: pb_util: avoid repeated stat() calls reading files
......................................................................
pb_util: avoid repeated stat() calls reading files
This reduces the number of fstat syscalls while loading a host with 11M blocks
from 29.3M to 147K.
Note that this also changes Env to return EndOfFile when reading from
disk rather than IOError as it used to.
Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
---
M src/kudu/util/env_posix.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
4 files changed, 58 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/8010/3
--
To view, visit http://gerrit.cloudera.org:8080/8010
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I27371800604bcb20bafae7946d3b3e84af094598
Gerrit-Change-Number: 8010
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins