You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2017/05/12 23:25:36 UTC
[kudu-CR] block manager: require file cache
Adar Dembo has uploaded a new change for review.
http://gerrit.cloudera.org:8080/6880
Change subject: block manager: require file cache
......................................................................
block manager: require file cache
The file cache has been in Kudu since 1.2 and we've yet to see any major
issues. So let's make it required; doing so lets us clean up some branches.
As for block_manager_max_open_files, I changed the meaning of the value 0
from "no limit" to "error"; this seemed net less disruptive than changing
the meaning of -1 from "40%" to "error" and 0 from "no limit" to "40%".
Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
---
M src/kudu/fs/block_manager.cc
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/file_block_manager.h
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
5 files changed, 53 insertions(+), 117 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/6880/1
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
[kudu-CR] block manager: require file cache
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: block manager: require file cache
......................................................................
Patch Set 2: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/6880/1/src/kudu/fs/block_manager.cc
File src/kudu/fs/block_manager.cc:
PS1, Line 48: if (value == 0) {
: LOG(ERROR) << "Invalid max open files: cannot be 0";
: return false;
: }
: return true;
> Dan asked me that when I was first committing the file cache code. Specific
sounds good
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] block manager: require file cache
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: block manager: require file cache
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/6880/1/src/kudu/fs/block_manager.cc
File src/kudu/fs/block_manager.cc:
PS1, Line 48: if (value == 0) {
: LOG(ERROR) << "Invalid max open files: cannot be 0";
: return false;
: }
: return true;
you think there's any value in making sure it's at least something semi-reasonable like 100?
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] block manager: require file cache
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/6880
to look at the new patch set (#2).
Change subject: block manager: require file cache
......................................................................
block manager: require file cache
The file cache has been in Kudu since 1.2 and we've yet to see any major
issues. So let's make it required; doing so lets us clean up some branches.
As for block_manager_max_open_files, I changed the meaning of the value 0
from "no limit" to "error"; this seemed net less disruptive than changing
the meaning of -1 from "40%" to "error" and 0 from "no limit" to "40%".
Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
---
M src/kudu/fs/block_manager.cc
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/file_block_manager.h
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
5 files changed, 54 insertions(+), 100 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/6880/2
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] block manager: require file cache
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: block manager: require file cache
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/6880/1/src/kudu/fs/block_manager.cc
File src/kudu/fs/block_manager.cc:
PS1, Line 48: if (value == 0) {
: LOG(ERROR) << "Invalid max open files: cannot be 0";
: return false;
: }
: return true;
> you think there's any value in making sure it's at least something semi-rea
Dan asked me that when I was first committing the file cache code. Specifically he was wondering whether performance fell off a cliff when max_open_files was below a certain amount.
The short answer is that I don't know; it'd depend a lot on what kind of workloads were being run. I also don't think it's worth worrying about since the gflag is already rather obscured.
If I'm wrong and it turns out that users use this as a footgun more often than not, I'll run some workloads to establish a reasonable minimum and add it to the validator.
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] block manager: require file cache
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged.
Change subject: block manager: require file cache
......................................................................
block manager: require file cache
The file cache has been in Kudu since 1.2 and we've yet to see any major
issues. So let's make it required; doing so lets us clean up some branches.
As for block_manager_max_open_files, I changed the meaning of the value 0
from "no limit" to "error"; this seemed net less disruptive than changing
the meaning of -1 from "40%" to "error" and 0 from "no limit" to "40%".
Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Reviewed-on: http://gerrit.cloudera.org:8080/6880
Reviewed-by: Todd Lipcon <to...@apache.org>
Tested-by: Kudu Jenkins
---
M src/kudu/fs/block_manager.cc
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/file_block_manager.h
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
5 files changed, 54 insertions(+), 100 deletions(-)
Approvals:
Todd Lipcon: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/6880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I9946e537e0b4abd66a9a90fc05df04232db68bc0
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>