You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2019/09/17 05:36:47 UTC
[kudu-CR] tools: don't open block manager when dumping UUID
Hello Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/14243
to review the following change.
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_fs.cc
3 files changed, 32 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/14243/1
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14243
to look at the new patch set (#2).
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
4 files changed, 34 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/14243/2
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14243/3/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/14243/3/src/kudu/tools/kudu-tool-test.cc@1771
PS3, Line 1771: fs.DumpFileSystemTree(tree_out);
> error: 'DumpFileSystemTree' is a private member of 'kudu::FsManager' [clang
Done
http://gerrit.cloudera.org:8080/#/c/14243/3/src/kudu/tools/tool_action_fs.cc
File src/kudu/tools/tool_action_fs.cc:
http://gerrit.cloudera.org:8080/#/c/14243/3/src/kudu/tools/tool_action_fs.cc@320
PS3, Line 320: fs_manager.DumpFileSystemTree(std::cout);
> error: 'DumpFileSystemTree' is a private member of 'kudu::FsManager' [clang
Done
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 17 Sep 2019 21:44:57 +0000
Gerrit-HasComments: Yes
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14243
to look at the new patch set (#4).
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
This also moves some words around in the header to try clarifying what
functions are important for startup.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
4 files changed, 60 insertions(+), 34 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/14243/4
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14243
to look at the new patch set (#3).
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
This also moves some words around in the header to try clarifying what
functions are important for startup.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
4 files changed, 60 insertions(+), 34 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/14243/3
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
PS2:
Could you adjust the function and/or class comments so that it's clearer which of Init/Open/LoadAndVerifyMetadataFiles/CreateInitialFileSystemLayout should be called? At least for the two most important use cases (i.e. creating a new FS and opening an existing one).
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h@156
PS2, Line 156:
: // Loads the instance metadata files and verifies that they are all matching,
: // returning any root paths that do not have metadata files.
Not clear what FsManager functions are safe to call after calling this and which aren't. Could you clarify in the comments?
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 17 Sep 2019 18:53:41 +0000
Gerrit-HasComments: Yes
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
This also moves some words around in the header to try clarifying what
functions are important for startup.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Reviewed-on: http://gerrit.cloudera.org:8080/14243
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
4 files changed, 55 insertions(+), 28 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
PS2:
> Could you adjust the function and/or class comments so that it's clearer wh
I only singled out Open() and CreateInitialFileSystemLayout(). Anything else seems like a bit too much implementation detail.
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h@156
PS2, Line 156: // Returns an error if the file system is already initialized.
: Status CreateInitialFileSystemLayout(
: boost::optional<std::string> uuid = boost::none);
> Not clear what FsManager functions are safe to call after calling this and
Done
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 17 Sep 2019 21:42:34 +0000
Gerrit-HasComments: Yes
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
PS2:
> I don't think this is nearly enough:
We chatted about this on slack; rather than adding two extra initialization-related methods and confusing developers that much more, this now adds a single one to partially Open() the FsManager.
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 17 Sep 2019 22:13:12 +0000
Gerrit-HasComments: Yes
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 1:
There isn't much behind this CR; I just ran the tool on a pretty loaded server and it took a while to get the UUID.
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 17 Sep 2019 05:56:00 +0000
Gerrit-HasComments: No
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 5: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14243/5/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/14243/5/src/kudu/fs/fs_manager.h@151
PS5, Line 151: Status PartialOpen(CanonicalizedRootsList* missing_roots = nullptr);
I might have called it OpenPartial but that's bike-shedding.
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 17 Sep 2019 22:15:18 +0000
Gerrit-HasComments: Yes
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14243
to look at the new patch set (#5).
Change subject: tools: don't open block manager when dumping UUID
......................................................................
tools: don't open block manager when dumping UUID
The block manager can be expensive to open, even when opening it in
read-only mode. If all a user wants is the server's UUID, all we really
need is to read the instance metadata files.
This patch updates the `kudu fs dump uuid` tool to do just that.
This also moves some words around in the header to try clarifying what
functions are important for startup.
Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_fs.cc
4 files changed, 55 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/14243/5
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] tools: don't open block manager when dumping UUID
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14243 )
Change subject: tools: don't open block manager when dumping UUID
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14243/2/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
PS2:
> I only singled out Open() and CreateInitialFileSystemLayout(). Anything els
I don't think this is nearly enough:
- Do I need to call Init() before calling Open() or CreateInitialFileSystemLayout()? Or do those functions do that for me?
- Likewise, do I need to call LoadAndVerifyMetadataFiles() before either of those other two? Do I need to call Init() first?
Taking a step back, FsManager now has four public functions whose names vaguely hint at affecting object lifecycle. When we only had two, the only ambiguity was whether to call Open after calling Create (it was somewhat obvious that Create didn't make sense after Open). Now there's a lot more ambiguity, especially with Init(), which sounds like it's always necessary.
--
To view, visit http://gerrit.cloudera.org:8080/14243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0acf3b08fc13c169a52d4cfe847efd62bd30cc0a
Gerrit-Change-Number: 14243
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 17 Sep 2019 21:48:45 +0000
Gerrit-HasComments: Yes