You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yingchun Lai (Code Review)" <ge...@cloudera.org> on 2022/12/14 12:58:51 UTC
[kudu-CR] [tools] Add 'kudu local replica edit delete rowsets' to delete rowsets from tablet
Yingchun Lai has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19357
Change subject: [tools] Add 'kudu local_replica edit delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica edit delete_rowsets' to delete rowsets from tablet
There are some cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corruption in a single point cluster, the server cannot be
started.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3376).
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard when
the tablet-metadata is dozens of megabyte.
This patch introduce a CLI tool 'kudu local_replica edit delete_rowsets' to delete
rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 206 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/1
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#5).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3376).
Note: no data will be lost in this case.
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
operate when the tablet-metadata is dozens of megabytes.
This patch introduces a CLI tool 'kudu local_replica tmeta delete_rowsets'
to delete rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 255 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/5
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 6: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Mon, 02 Jan 2023 08:48:10 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yuqi Du (Code Review)" <ge...@cloudera.org>.
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 6: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@105
PS3, Line 105: DEFINE_bool(backup_metadata, false,
> > It's safer using default 'true' and use 'enable_adding_orphaned_blocks' f
The CLI command is very useful for the replicaion_factor = 1, and the only one tablet replica is corrupted.
It seems the CLI command should work at FLAGS_backup_metadata false and FLAGS_enable_adding_orphaned_blocks true, and after running it. And it is impossible to rollback. Hence, serious problems may happen by incautious administrators.
In order to avoid the serious problems, backup metadata/data may needed. But it's a little difficult because the deleting operation can not avoid some dangerous. At the same time, the CLI command face to a tserver, if a tablet has 3 replicas, it should keep writing stop and operate at 3 tservers.
Maybe we should add some describe words or some forbidden condition later.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Tue, 03 Jan 2023 02:47:53 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 5: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19357/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/5//COMMIT_MSG@13
PS5, Line 13: 3376
nit: KUDU-3367
http://gerrit.cloudera.org:8080/#/c/19357/5/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/5/src/kudu/tools/kudu-tool-test.cc@8818
PS5, Line 8818: // Make sure there are some rowsets flushed, altering the table can achieved it.
nit: I do not see any alter table operations, may be this is irrelevant?
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Fri, 30 Dec 2022 06:57:42 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#4).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3376).
Note: no data will be lost in this case.
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
operate when the tablet-metadata is dozens of megabytes.
This patch introduces a CLI tool 'kudu local_replica tmeta delete_rowsets'
to delete rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 255 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/4
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, KeDeng,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#2).
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corruption in a single point cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3376).
Note: no data will be lost in this case.
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
operate when the tablet-metadata is dozens of megabytes.
This patch introduce a CLI tool 'kudu local_replica edit_metadata delete_rowsets'
to delete rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 255 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/2
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 7:
(4 comments)
Overall looks good to me, just a couple of nits.
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@158
PS7, Line 158: DECLARE_bool(enable_tablet_orphaned_block_deletion);
nit: is this still needed?
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@241
PS7, Line 241:
To promote a safer usage of this tool, consider changing the default value for the --enable_tablet_orphaned_block_deletion flag to 'false' in the kudu CLI binary by adding the following:
CHECK_NE("", SetCommandLineOptionWithMode(
"enable_tablet_orphaned_block_deletion",
"false",
SET_FLAGS_DEFAULT));
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@1266
PS7, Line 1266: (
How about the following text for ExtraDescription() with minor improvements on the original:
The common usage pattern of this tool is described below. That involves checking the result by a dry run of the tablet server with the modified tablet's data after running the tool. It's crucial to customize tablet server's --enable_tablet_orphaned_block_deletion flag for the dry run to avoid deleting orphaned blocks, so it's possible to roll back to the original state of the tablet's data if something goes wrong. First, run the tool with default settings for --backup_metadata and --enable_tablet_orphaned_block_deletion to (a) create a backup of the original metadata file and (b) keep the orphaned blocks on the file system. Second, start the tablet server with --enable_tablet_orphaned_block_deletion=false to check whether the change worked as expected and the tablet server works fine with the new state of the tablet's data. If it doesn't work as expected, stop the tablet server (if still running), rollback the change by replacing the updated metadata file with the backup created earlier, and retry the procedure again, specifying proper rowset identifiers to the tool. If the change works as expected and the tablet server runs fine after with the updated tablet's data, remove the customization for the --enable_tablet_orphaned_block_deletion flag and restart the tablet server.
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@1269
PS7, Line 1269: "
nit: add a space before the double quote
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Wed, 04 Jan 2023 20:44:17 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 6:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19357/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/5//COMMIT_MSG@13
PS5, Line 13: 3367
> nit: KUDU-3367
Done
http://gerrit.cloudera.org:8080/#/c/19357/5/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/5/src/kudu/tools/kudu-tool-test.cc@8818
PS5, Line 8818: // Make sure there are some rowsets flushed.
> nit: I do not see any alter table operations, may be this is irrelevant?
Done
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Fri, 30 Dec 2022 14:43:33 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 7:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@16
PS6, Line 16: There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
: hard to operate when working with large amount of data.
> There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
Done
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@19
PS6, Line 19: a
> a new
Done
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@20
PS6, Line 20: which makes removing rowsets from a tablet much easier.
> which makes removing rowsets from a tablet much easier
Done
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tablet/tablet_metadata.cc@68
PS6, Line 68:
> Instead of another tricky flag that might corrupt the metadata (in the sens
It's not enough safe and convenient if just disable --enable_tablet_orphaned_block_deletion. Without the new added --enable_tablet_orphaned_block_adding, after blocks have been added to 'orphaned_blocks_', they will be persisted to super block PB file. So we have to change --enable_tablet_orphaned_block_deletion to false (which is not the default value) when use the CLI tool and the next time when restart server to check if the operation is expected.
If some one forget to chenge the default value when restart the server, blocks will be deleted and the backup file is useless then.
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@a6028
PS6, Line 6028:
> nit: why is this change? I'd suggest making changes that are relevant only
Agree, done.
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@8844
PS6, Line 8844: metadata_pa
> nit: original_file
Done
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@8898
PS6, Line 8898: LSE(backup_f
> nit: 'no data is lost' or 'there isn't any data loss'
Done
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@106
PS6, Line 106: edit
> editing
Done
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@640
PS6, Line 640: if (FLAGS_backup_metadata) {
: // Move the old tablet metadata file to a backup location.
: string original_path = fs_manager.GetTabletMetadataPath(tablet_id);
: string backup_path = Substitute("$0.bak.$1", original_path, GetCurrentTimeMicros());
: RETURN_NOT_OK_PREPEND(Env::Default()->RenameFile(original_path, backup_path),
: "couldn't back up original file");
: LOG(INFO) << "Moved original file to " << backup_path;
: }
:
> Maybe, move this input verification to be after other checks at lines 622-6
Good idea.
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@662
PS6, Line 662: s::InvalidArgument("no tablet
> nit: Successfully removed rowsets with identifiers:
Done
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@1276
PS6, Line 1276: ionalParameter("enable_tablet_orpha
> I'd suggest to use already existing --enable_tablet_orphaned_block_deletion
Emm, OK, add a more tricky flag maybe not reasonable enough, Idecide to remove the new added flag after trade-off the two choices. And it's able to enable --backup_metadata by default, which Yuqi suggest.
When administrators input some incorrect rowset ids, it works not as expected.
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@1275
PS6, Line 1275:
: .AddOptionalParameter("enable_tablet_orphaned_block_deletion")
: .AddOptionalParameter("fs_data_dirs")
: .AddOptionalParameter("fs_metadata_dir")
: .AddOptionalParameter("fs_wal_dir")
: .Build();
:
> Move this into ExtraDescription(). See https://github.com/apache/kudu/blob
Done
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Wed, 04 Jan 2023 16:57:19 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3367).
Note: no data will be lost in this case.
There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
hard to operate when working with large amount of data.
This patch introduces a new CLI tool 'kudu local_replica tmeta delete_rowsets'
which makes removing rowsets from a tablet much easier.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Reviewed-on: http://gerrit.cloudera.org:8080/19357
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <al...@apache.org>
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 251 insertions(+), 2 deletions(-)
Approvals:
Kudu Jenkins: Verified
Alexey Serbin: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 10
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
(9 comments)
> Patch Set 3:
>
> (1 comment)
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG@7
PS1, Line 7: edit
> Done
Renamed to tmeta at last, thanks.
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@7
PS3, Line 7: edit_metadata
> what about 'edit_tmeta'
Renamed to tmeta, thanks.
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@11
PS3, Line 11: corruption
> corrupted
Done
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@11
PS3, Line 11: a single point cluster
> What's 'a single point cluster'? Probably, you meant 'a single node cluste
Yes, thanks for your correction.
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@19
PS3, Line 19: introduce
> introduces
Done
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tablet/tablet_metadata.cc@76
PS3, Line 76: "tablet-metadata to roll back. Note: This is only used for CLI "
> It seems a little complex.
Good advice. I moved most description to the tool description.
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@105
PS3, Line 105: DEFINE_bool(backup_metadata, false,
> It's safer using default 'true' and use 'enable_adding_orphaned_blocks' false ?
--enable_adding_orphaned_blocks should be true by default, it's the safer value for servers. But it's conflict when using CLI tools when enable --backup_metadata. So set --backup_metadata default value as false. I think it's reasonable because backup is just an optional manner.
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@1250
PS3, Line 1250: edit_matadata
> Could we have a sub-command 'delete_rowsets' for a new 'metadata' or 'tmeta
Renamed to tmeta, thanks.
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@1250
PS3, Line 1250: ModeBuilder("edit_matadata")
> What about renaming ’metadata‘ to 'tmeta'
Renamed to tmeta, thanks.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Wed, 28 Dec 2022 16:18:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 8:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@158
PS7, Line 158: DECLARE_int32(num_threads);
> nit: is this still needed?
Not needed, removed.
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@241
PS7, Line 241: class TabletCopier {
> To promote a safer usage of this tool, consider changing the default value
Got it, thanks.
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@1266
PS7, Line 1266: s
> How about the following text for ExtraDescription() with minor improvements
Thanks a lot.
http://gerrit.cloudera.org:8080/#/c/19357/7/src/kudu/tools/tool_action_local_replica.cc@1269
PS7, Line 1269:
> nit: add a space before the double quote
Done
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc
File src/kudu/tools/tool_main.cc:
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc@260
PS8, Line 260: CHECK_NE("", google::SetCommandLineOptionWithMode(
Reset the default value before calling gflags::ParseCommandLineNonHelpFlags, then we can get the correct new default value when type --help.
Some other existing SetCommandLineOptionWithMode calls are placed after ParseCommandLineNonHelpFlags, we can improve them in another patch.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Thu, 05 Jan 2023 17:21:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 6:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@16
PS6, Line 16: Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
: operate when the tablet-metadata is dozens of megabytes.
There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and hard to operate when working with large amount of data.
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@19
PS6, Line 19: a
a new
http://gerrit.cloudera.org:8080/#/c/19357/6//COMMIT_MSG@20
PS6, Line 20: to delete rowsets from a tablet, which is easier to use
which makes removing rowsets from a tablet much easier
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tablet/tablet_metadata.cc@68
PS6, Line 68: enable_tablet_orphaned_block_adding
Instead of another tricky flag that might corrupt the metadata (in the sense of not tracking the blocks to be deleted), maybe use already existing flag --enable_tablet_orphaned_block_deletion?
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@a6028
PS6, Line 6028:
nit: why is this change? I'd suggest making changes that are relevant only to the essence of the patch, otherwise this sort of a change might result in an extra conflict when trying to cherry-pick this.
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@8844
PS6, Line 8844: origin_file
nit: original_file
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/kudu-tool-test.cc@8898
PS6, Line 8898: no data lost
nit: 'no data is lost' or 'there isn't any data loss'
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@106
PS6, Line 106: edit
editing
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@640
PS6, Line 640: RowSetMetadataIds to_remove;
: for (const auto& rowset_id_str : rowset_ids_vec) {
: int64_t rowset_id;
: if (safe_strto64(rowset_id_str.c_str(), &rowset_id)) {
: to_remove.insert(rowset_id);
: } else {
: return Status::InvalidArgument(Substitute("$0 is not a valid rowset id.", rowset_id_str));
: }
: }
Maybe, move this input verification to be after other checks at lines 622-627 and before opening FSManager?
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@662
PS6, Line 662: Succeed to remove rowset ids:
nit: Successfully removed rowsets with identifiers:
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@1276
PS6, Line 1276: enable_tablet_orphaned_block_adding
I'd suggest to use already existing --enable_tablet_orphaned_block_deletion flag here for both running the tool and then for a dry-run of kudu-tserver to verify that the operation worked as expected.
BTW, what are conditions when the 'delete_rowsets' works not as expected?
http://gerrit.cloudera.org:8080/#/c/19357/6/src/kudu/tools/tool_action_local_replica.cc@1275
PS6, Line 1275: You can set "
: "--enable_tablet_orphaned_block_adding to false, then only "
: "rowset ids are removed from tablet metadata, the related blocks "
: "are kept on file system. You can start the server to check "
: "whether the operation is worked as expect, if it is, then "
: "shutdown the server and run 'kudu fs check --repair' to remove "
: "orphaned blocks, or use the backup tablet-metadata to roll back.")
Move this into ExtraDescription(). See https://github.com/apache/kudu/blob/986f1a0cd0427e548a8bff7ccc53b626335ebc57/src/kudu/tools/tool_action_remote_replica.cc#L473-L479 for an example.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Wed, 04 Jan 2023 02:39:30 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Wang Xixu (Code Review)" <ge...@cloudera.org>.
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
> Patch Set 3: Verified+1
>
> Build Successful
>
> http://jenkins.kudu.apache.org/job/kudu-gerrit/26904/ : SUCCESS
+1
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 01:18:11 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#6).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3367).
Note: no data will be lost in this case.
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
operate when the tablet-metadata is dozens of megabytes.
This patch introduces a CLI tool 'kudu local_replica tmeta delete_rowsets'
to delete rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 255 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/6
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 9: Code-Review+2
Thank you for the contribution!
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 17:44:29 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] Add 'kudu local replica edit delete rowsets' to delete rowsets from tablet
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG@7
PS1, Line 7: edit
nit: How about renaming it to 'edit_metadata'?
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc@71
PS1, Line 71: only rowset ids are removed from "
: "tablet metadata, the related blocks are kept on file system. You "
: "can start the server to check whether the operation is worked as "
: "expect, if it is, then shutdown the server and run 'kudu fs check "
: "--repair=true' to remove orphaned blocks, or use the backup "
: "tablet-metadata to roll back.
I'm curious about the scene where we need to keep blocks in this way. From my understanding, is it enough to use the backup flag?
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tools/kudu-tool-test.cc@8812
PS1, Line 8812: ASSERT_EQ(rowset_ids.size() - 1, new_rowset_ids.size());
nit: Should we also check that new_rowset_ids do not contain the id of deleted rowset?
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Thu, 15 Dec 2022 10:16:46 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit delete_rowsets' to delete rowsets from tablet
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#9).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3367).
Note: no data will be lost in this case.
There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
hard to operate when working with large amount of data.
This patch introduces a new CLI tool 'kudu local_replica tmeta delete_rowsets'
which makes removing rowsets from a tablet much easier.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 251 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/9
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#7).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3367).
Note: no data will be lost in this case.
There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
hard to operate when working with large amount of data.
This patch introduces a new CLI tool 'kudu local_replica tmeta delete_rowsets'
which makes removing rowsets from a tablet much easier.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 240 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/7
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc@71
PS1, Line 71: only rowset ids are removed from "
: "tablet metadata, the related blocks are kept on file system. You "
: "can start the server to check whether the operation is worked as "
: "expect, if it is, then shutdown the server and run 'kudu fs check "
: "--repair=true' to remove orphaned blocks, or use the backup "
: "tablet-metadata to roll back.
> Got it. Thanks for you explanation!
Because there is an existing flag --enable_tablet_orphaned_block_deletion above, --keep_orphaned_blocks may make users confused a bit.
How about rename to --enable_tablet_orphaned_block_adding?
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Sat, 24 Dec 2022 10:49:14 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 8:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc@63
PS8, Line 63: or
and
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc@64
PS8, Line 64: for
in
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc
File src/kudu/tools/tool_main.cc:
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc@260
PS8, Line 260: CHECK_NE("", google::SetCommandLineOptionWithMode(
> Reset the default value before calling gflags::ParseCommandLineNonHelpFlags
Ah, indeed -- it seems I misguided you on this topic, and the result is many failed tests.
I think this should be done a different way in the context of the CLI tool: see https://github.com/apache/kudu/blob/986f1a0cd0427e548a8bff7ccc53b626335ebc57/src/kudu/tools/tool_action_master.cc#L815 for example. I guess we need to have similar setting for "enable_tablet_orphaned_block_deletion" at https://gerrit.cloudera.org/#/c/19357/8/src/kudu/tools/tool_action_local_replica.cc@1285
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Thu, 05 Jan 2023 19:56:22 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 9:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc@63
PS8, Line 63: an
> and
Done
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tablet/tablet_metadata.cc@64
PS8, Line 64: in
> in
Done
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc
File src/kudu/tools/tool_main.cc:
http://gerrit.cloudera.org:8080/#/c/19357/8/src/kudu/tools/tool_main.cc@260
PS8, Line 260: //
> Ah, indeed -- it seems I misguided you on this topic, and the result is man
Done
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 15:31:15 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yuqi Du (Code Review)" <ge...@cloudera.org>.
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@7
PS3, Line 7: edit_metadata
what about 'edit_tmeta'
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tablet/tablet_metadata.cc@76
PS3, Line 76: "tablet-metadata to roll back. Note: This is only used for CLI "
It seems a little complex.
How to use the tool an the series operations, it's better write these into docs in the future
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@105
PS3, Line 105: DEFINE_bool(backup_metadata, false,
It's safer using default 'true' and use 'enable_adding_orphaned_blocks' false ?
The tool's aim is setting the two flags FLAGS_backup_metadata false and FLAGS_enable_adding_orphaned_blocks true, the setting is convenient for the fault case, isn't it?
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@1250
PS3, Line 1250: ModeBuilder("edit_matadata")
What about renaming ’metadata‘ to 'tmeta'
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 12:28:59 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#3).
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corruption in a single point cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3376).
Note: no data will be lost in this case.
Although we can use 'kudu pbc edit' to achieve this aim, but it's very hard to
operate when the tablet-metadata is dozens of megabytes.
This patch introduce a CLI tool 'kudu local_replica edit_metadata delete_rowsets'
to delete rowsets from a tablet, which is easier to use.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
5 files changed, 255 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/3
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Yuqi Du, Yifan Zhang, Kudu Jenkins, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19357
to look at the new patch set (#8).
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
[tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
There are some use cases we need to delete rowsets from a tablet.
For example：
1. Some blocks are corrupted in a single node cluster, the server cannot be
started. Note: some data will be lost in this case.
2. Some rowsets are fully deleted but the blocks can not be GCed (KUDU-3367).
Note: no data will be lost in this case.
There is 'kudu pbc edit' CLI tool to achieve that, but it's error prone and
hard to operate when working with large amount of data.
This patch introduces a new CLI tool 'kudu local_replica tmeta delete_rowsets'
which makes removing rowsets from a tablet much easier.
Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
---
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tools/tool_main.cc
6 files changed, 255 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/19357/8
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@11
PS3, Line 11: a single point cluster
What's 'a single point cluster'? Probably, you meant 'a single node cluster'?
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@11
PS3, Line 11: corruption
corrupted
http://gerrit.cloudera.org:8080/#/c/19357/3//COMMIT_MSG@19
PS3, Line 19: introduce
introduces
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@1250
PS3, Line 1250: edit_matadata
Could we have a sub-command 'delete_rowsets' for a new 'metadata' or 'tmeta' command? Keeping the commands hierarchically structured helps to find the required functionality easier for an operator. Also, it's more in-line with the approach used for the 'cmeta' command.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Wed, 28 Dec 2022 03:27:52 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc@71
PS1, Line 71: only rowset ids are removed from "
: "tablet metadata, the related blocks are kept on file system. You "
: "can start the server to check whether the operation is worked as "
: "expect, if it is, then shutdown the server and run 'kudu fs check "
: "--repair=true' to remove orphaned blocks, or use the backup "
: "tablet-metadata to roll back.
> Without the flag, the rowset blocks will be added to orphaned_blocks_ (by A
Got it. Thanks for you explanation!
Maybe we can define this flag 'keep_orphaned_blocks' with a default value 'false', so we can use '--keep_orphaned_blocks' in case we do not want to delete blocks, which is more intuitive.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Thu, 22 Dec 2022 03:53:43 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica tmeta delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica tmeta delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19357/3/src/kudu/tools/tool_action_local_replica.cc@105
PS3, Line 105: DEFINE_bool(backup_metadata, false,
> The CLI command is very useful for the replicaion_factor = 1, and the only
I agree that dangerous operation should be added enough tips and provide rollback methods, and this patch do it like this indeed.
Administrators should always know what they are doing when run CLI tools, some 'dangerous' operations like drop tables, delete columns, edit cmeta, edit pbc files, and etc, no one force you to add any backup flags. IMO, backup is optional but not a force operation.
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Tue, 03 Jan 2023 10:28:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add 'kudu local replica edit metadata delete rowsets' to delete rowsets from tablet
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19357 )
Change subject: [tools] Add 'kudu local_replica edit_metadata delete_rowsets' to delete rowsets from tablet
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19357/1//COMMIT_MSG@7
PS1, Line 7: edit
> nit: How about renaming it to 'edit_metadata'?
Done
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc
File src/kudu/tablet/tablet_metadata.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tablet/tablet_metadata.cc@71
PS1, Line 71: only rowset ids are removed from "
: "tablet metadata, the related blocks are kept on file system. You "
: "can start the server to check whether the operation is worked as "
: "expect, if it is, then shutdown the server and run 'kudu fs check "
: "--repair=true' to remove orphaned blocks, or use the backup "
: "tablet-metadata to roll back.
> I'm curious about the scene where we need to keep blocks in this way. From
Without the flag, the rowset blocks will be added to orphaned_blocks_ (by AddOrphanedBlocksUnlocked, in line 693), the follow up Flush will delete the blocks from fs_manager (line 592), so even if we can use the backup metadata to try to roll back, the blocks have been removed. Then some errors will report when restart server.
Failed to open rowset RowSet(0): Not found: Can't find block: 0000000000000036
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19357/1/src/kudu/tools/kudu-tool-test.cc@8812
PS1, Line 8812: ASSERT_EQ(rowset_ids.size() - 1, new_rowset_ids.size());
> nit: Should we also check that new_rowset_ids do not contain the id of dele
Done
--
To view, visit http://gerrit.cloudera.org:8080/19357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If2cf9035babf4c3af4c238cebe8dcecd2c65848f
Gerrit-Change-Number: 19357
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: KeDeng <kd...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>
Gerrit-Comment-Date: Fri, 16 Dec 2022 17:08:04 +0000
Gerrit-HasComments: Yes