You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "wangning (Code Review)" <ge...@cloudera.org> on 2020/03/06 14:23:26 UTC
[kudu-CR] KUDU-3070 allow rewrite raft config take multi tablet id as input
wangning has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15380
Change subject: KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
......................................................................
KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
There are some migration cases which need multi tablet_id as input
1. migrate data from standalone node another standalone node
2. migrate data from 3-tservers cluster to another 3-tservers cluster
3. migrate data for those tablet with same raft peers
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/tools/tool_action_local_replica.cc
1 file changed, 35 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/1
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 1
Gerrit-Owner: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 13:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@134
PS12, Line 134: // Whether or not to skip opening the block manager. FsManager operations that
> Nit: "Whether or not to skip opening the block manager. FsManager operation
Done
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@138
PS12, Line 138: bool skip_block_manager;
> Nit: add an extra empty line after this.
Done
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@293
PS12, Line 293:
> Should DCHECK here as well.
Done
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/tools/tool_action_local_replica.cc@289
PS12, Line 289: FsManager fs_manager(env, std::move(fs_opts));
> Should std::move(fs_opts).
Done
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 13
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Mar 2020 06:33:54 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#13).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 23 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/13
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 13
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc@145
PS3, Line 145: skip_block_manager(false) {
> Can you go through the various public FsManager methods that reference bloc
It take me sometime to fully understand it.
Didn't realized new mechanism introduced may hurt current system.
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc@435
PS3, Line 435:
:
: /
> I don't think we need to log this; could just drop it.
Done
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 7
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Mon, 09 Mar 2020 08:56:22 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#5).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 21 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/5
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 5
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#14).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 23 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/14
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 14
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#3).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 17 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/3
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 3
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#12).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 21 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/12
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 12
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#11).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 22 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/11
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 11
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 4:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.h@134
PS3, Line 134: // Whether or not skip open the block manager.
Nit: trailing white space and an extra space before the period.
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc@145
PS3, Line 145: skip_block_manager(false) {
Can you go through the various public FsManager methods that reference block_manager_ and add DCHECK(block_manager_) to them?
http://gerrit.cloudera.org:8080/#/c/15380/3/src/kudu/fs/fs_manager.cc@435
PS3, Line 435: else {
: LOG(INFO) << "Skipped open block manager";
: }
I don't think we need to log this; could just drop it.
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 4
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Mon, 09 Mar 2020 04:33:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#6).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 21 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/6
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 6
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#9).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 23 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/9
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 9
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Reviewed-on: http://gerrit.cloudera.org:8080/15380
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 23 insertions(+), 8 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
Andrew Wong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 15
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 14: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 14
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Mar 2020 21:41:34 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3070 allow rewrite raft config take multi tablet id as input
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
......................................................................
Patch Set 2:
> Patch Set 2:
>
> While reviewing this, I naturally wondered why we should allow the list to receive a list of tablet IDs rather than force users to provide the list using a shell script loop. Then I looked at what you had written in KUDU-3070 and saw that you had began with that, but the repeated cost of opening an FsManager was causing the overall operation to be very expensive.
>
> How about this as an alternative: let's make it possible to open an FsManager _without opening a BlockManager_. There are many CLI tools that don't manipulate data blocks and could use this alternate open mode to speed up their operations. But by opening the FsManager and DataDirManager, we continue to enforce that read-write operations must lock the directories first, preventing consistency issues if run alongside a running tserver/master.
>
> To make this work, we'd need to skip the following lines of code in fs_manager.cc:
>
> // Finally, initialize and open the block manager.
> InitBlockManager();
> LOG_TIMING(INFO, "opening block manager") {
> RETURN_NOT_OK(block_manager_->Open(report));
> }
>
> We'd also need to make sure that the returned FsReport makes sense, and also that the various FsManager methods that invoke or return the BlockManager DCHECK that it exists first. Then we can start rolling out this alternate open mode for whichever CLI tools could safely use it.
>
> What do you think?
That's true, the consistency for tablets is not easy to guarantee.
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 2
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Mon, 09 Mar 2020 02:35:43 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 14: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 14
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Mar 2020 06:42:52 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#4).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 18 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/4
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 4
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#7).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 20 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/7
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 7
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.h@134
PS8, Line 134: open
nit: "opening"
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.cc@148
PS8, Line 148:
: FsManagerOpts::FsManagerOpts(const string& root)
: : wal_root(root),
: data_roots({ root }),
: block_manager_type(FLAGS_block_manager),
: read_only(false),
: update_instances(UpdateInstanceBehavior::UPDATE_AND_IGNORE_FAILURES),
: file_cache(nullptr) {}
:
: FsManager::FsManager(Env* env, FsManagerOpts opts)
: : env_(DCHECK_NOTNULL(env)),
: opts_(std::move(opts)),
: error_manager_(new FsErrorManager()),
: initted_(false) {
: DCHECK(opts_.update_instances == UpdateInstanceBehavior::DONT_UPDATE ||
: !opts_.read_only) << "FsManager can only be for updated if not in read-only mode";
: }
These constructors will have undefined behavior since skip_block_manager isn't initialized.
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 8
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Mon, 09 Mar 2020 20:48:04 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3070 allow rewrite raft config take multi tablet id as input
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#2).
Change subject: KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
......................................................................
KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
There are some migration cases which need multi tablet_id as input
1. migrate data from standalone node another standalone node
2. migrate data from 3-tservers cluster to another 3-tservers cluster
3. migrate data for those tablet with same raft peers
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/tools/tool_action_local_replica.cc
1 file changed, 35 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/2
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 2
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#10).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 19 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/10
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 10
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 12:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@134
PS12, Line 134: // Whether or not skip opening the block manager.
Nit: "Whether or not to skip opening the block manager. FsManager operations that require the block manager will crash."
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@138
PS12, Line 138: };
Nit: add an extra empty line after this.
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/fs/fs_manager.h@293
PS12, Line 293: return block_manager_.get();
Should DCHECK here as well.
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/15380/12/src/kudu/tools/tool_action_local_replica.cc@289
PS12, Line 289: FsManager fs_manager(env, fs_opts);
Should std::move(fs_opts).
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 12
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Mar 2020 06:20:25 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3070 allow rewrite raft config take multi tablet id as input
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 allow rewrite_raft_config take multi tablet_id as input
......................................................................
Patch Set 2:
While reviewing this, I naturally wondered why we should allow the list to receive a list of tablet IDs rather than force users to provide the list using a shell script loop. Then I looked at what you had written in KUDU-3070 and saw that you had began with that, but the repeated cost of opening an FsManager was causing the overall operation to be very expensive.
How about this as an alternative: let's make it possible to open an FsManager _without opening a BlockManager_. There are many CLI tools that don't manipulate data blocks and could use this alternate open mode to speed up their operations. But by opening the FsManager and DataDirManager, we continue to enforce that read-write operations must lock the directories first, preventing consistency issues if run alongside a running tserver/master.
To make this work, we'd need to skip the following lines of code in fs_manager.cc:
// Finally, initialize and open the block manager.
InitBlockManager();
LOG_TIMING(INFO, "opening block manager") {
RETURN_NOT_OK(block_manager_->Open(report));
}
We'd also need to make sure that the returned FsReport makes sense, and also that the various FsManager methods that invoke or return the BlockManager DCHECK that it exists first. Then we can start rolling out this alternate open mode for whichever CLI tools could safely use it.
What do you think?
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 2
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Sun, 08 Mar 2020 22:32:05 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15380
to look at the new patch set (#8).
Change subject: KUDU-3070 skip open block manager
......................................................................
KUDU-3070 skip open block manager
In some cli ops (e.g. local_replica cmeta rewrite_raft_config), open
block manager is not necessary.
Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/tools/tool_action_local_replica.cc
3 files changed, 19 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/15380/8
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 8
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
[kudu-CR] KUDU-3070 skip open block manager
Posted by "wangning (Code Review)" <ge...@cloudera.org>.
wangning has posted comments on this change. ( http://gerrit.cloudera.org:8080/15380 )
Change subject: KUDU-3070 skip open block manager
......................................................................
Patch Set 9:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.h@134
PS8, Line 134: open
> nit: "opening"
Done
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/15380/8/src/kudu/fs/fs_manager.cc@148
PS8, Line 148:
: FsManagerOpts::FsManagerOpts(const string& root)
: : wal_root(root),
: data_roots({ root }),
: block_manager_type(FLAGS_block_manager),
: read_only(false),
: update_instances(UpdateInstanceBehavior::UPDATE_AND_IGNORE_FAILURES),
: file_cache(nullptr),
: skip_block_manager(false) {}
:
: FsManager::FsManager(Env* env, FsManagerOpts opts)
: : env_(DCHECK_NOTNULL(env)),
: opts_(std::move(opts)),
: error_manager_(new FsErrorManager()),
: initted_(false),
: skip_block_manager(false) {
:
> These constructors will have undefined behavior since skip_block_manager is
Done
--
To view, visit http://gerrit.cloudera.org:8080/15380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaec03512086430a6270c458269da1cf996fd9c5
Gerrit-Change-Number: 15380
Gerrit-PatchSet: 9
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: wangning <19...@gmail.com>
Gerrit-Comment-Date: Tue, 10 Mar 2020 01:28:36 +0000
Gerrit-HasComments: Yes