You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yingchun Lai (Code Review)" <ge...@cloudera.org> on 2019/03/26 03:40:06 UTC
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Yingchun Lai has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12852
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
[maintenance] Add privilege maintenance thread pool for privilege tables and tablets
This commit add a privilege thread pool in maintenance manager for
privilege tables and tablets.
In a Kudu cluster with thousands of tables and tablets, it's hard for
a specified tablet's maintenance OPs to be launched when their scores
are not the highest, even if the table the tablet belongs to is high
priority for Kudu users. This patch allow user to specify privilege
tables and tablets by gflags, these maintenance OPs of these privilege
tables and tablets can be launched in a privilege thread pool, so they
can have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/tserver/tserver_path_handlers.cc
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
M www/maintenance-manager.mustache
15 files changed, 445 insertions(+), 154 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/1
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
> Patch Set 1:
>
> > 3. Like rocksDB, we keep two kind of thread pools, one for flush, and the other for compaction(and GC in Kudu).
>
> How does this work? Are priority multipliers only applied to the compaction/GC pool?
>
> Separately, if the goal is to do some sort of resource management on tablets, shouldn't we also consider prioritization at the point of ingest (i.e. during a Write RPC)? By only considering the MM we're potentially putting ourselves in difficult situations (e.g. a bunch of data can't be GC'ed because although it was written successfully, the tablet is "low" priority so the MM has deprioritized WAL GC for it).
Priority multipliers only applied to the compaction ops to improve performance.
Priority multipliers could not applied to GC ops and flush ops, so they can work like before.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Wed, 08 May 2019 02:33:02 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 7: Code-Review+2
Carrying forward Will's +2.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 21 May 2019 20:22:16 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 5:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/tablet/tablet_mm_ops.h
File src/kudu/tablet/tablet_mm_ops.h:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/tablet/tablet_mm_ops.h@62
PS5, Line 62: void UpdateStats(MaintenanceOpStats* stats) OVERRIDE;
> If you're already cleaning this up, you could also convert 'OVERRIDE' into
Done
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc
File src/kudu/util/maintenance_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@77
PS5, Line 77: void SetUp() OVERRIDE {
> Nit: 'override' is more C++-11 compliant.
Done
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@253
PS5, Line 253: op1.set_sleep_time(MonoDelta::FromMilliseconds(10));
> This doesn't seem look a good change: by going from a sleep time of 1000ms
I just want to reduce unit test runing time, seem I have to revert it.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h
File src/kudu/util/maintenance_manager.h:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h@233
PS5, Line 233: virtual const std::string& table_id() const = 0;
> This is a different visibility than the other table_id definitions in this
It should be 'protected' as the others, I've updated it.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h@311
PS5, Line 311: typedef std::unordered_map<std::string, int32_t> TablePriorities;
> I assume the map keys are table names? That gets tricky when tables are ren
No, they are table IDs.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc
File src/kudu/util/maintenance_manager.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@a531
PS5, Line 531:
:
:
> I take it you dropped this because you didn't want to worry about locking t
No, this field is not used, I think we needn't keep it.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@100
PS5, Line 100: DEFINE_string(maintenance_manager_table_priorities, "",
: "Priorities of tables, semicolon-separated list of table-priority pairs, and each "
: "table-priority pair is combined by table id, colon and priority level. Priority "
: "level is ranged in [-FLAGS_max_priority_range, FLAGS_max_priority_range]");
: TAG_FLAG(maintenance_manager_table_priorities, advanced);
: TAG_FLAG(maintenance_manager_table_priorities, experimental);
: TAG_FLAG(maintenance_manager_table_priorities, runtime);
> I'm not really going to review this aspect of your patch, because I assume
I agree, no hurry, we can wait it.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@484
PS5, Line 484: if (best_perf_improvement_op && best_perf_improvement > 0) {
> Here too.
Done
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Sat, 18 May 2019 05:50:17 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 6: Code-Review+1
Let's see if we can't get another reviewer for this...
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Mon, 20 May 2019 03:42:29 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
The purpose of this feature is not only for more important user in multitenant use case, but also aim to reduce the negative influence of some tablets as quick as possible(the tablet may have more delta and RS overlaps, and more write and scan requests on it, which may cost more cpu, disk util, etc, which will pulled down the tserver performance).
This feature is more likely to be use as a manual compact tool when I start to design it, gflags should be reset after some minitues when the tablet layout seems health.
It seems not easy to confirm multiplier value, too high value will let other tablets less or no chance to be launched, and too low value may make no effect on the priority tablets.
It's more general to use multiple priority levels, higher level has higher multiplier value.
'extra config' is useful if we can change it dynamically.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 26 Mar 2019 11:24:58 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#7).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit adds a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allows administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 410 insertions(+), 155 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/7
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#3).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit add a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allow administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 404 insertions(+), 153 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/3
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
Some design improvements:
1. Define several integer priority levels for tables, like [-5, 5], default level is 0
2. Each level has a multiplier on the op scores, Level N's multiplier is base_multiplier^N, base_multiplier is some like 1.1
3. Like rocksDB, we keep two kind of thread pools, one for flush, and the other for compaction(and GC in Kudu).
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 07 May 2019 16:22:56 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#5).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit add a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allow administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 407 insertions(+), 153 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/5
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
A few questions about the design/feature:
It seems this patch implements a single "privileged" class for tables. Is this flexible enough for all scenarios? I can think of a couple of alternatives:
1) a "boost" which could be applied as a multiplier on top of the op scores for ops in a given tablet. This isn't a guaranteed priority, but can help to make sure that the high-priority tables get prioritized more quickly than others.
2) a more general priority scheme: I think forcing tables into a "normal/high priority" distinction is difficult to manage in a multitenant cluster. I'm afraid this will start getting used, and then at some point someone's going to say "oh, we need a SUPER-HIGH class", etc. Is it worth considering integer priorities?
3) some kind of fair-sharing or budgeting mechanism for maintenance ops across tablets. In other words, in a multitenant use case, we may want to ensure that each tenant gets a fair share of the MM thread pool's resources (perhaps allowing some kind of multiplier/allocation for tenants that are more important). This solves the use case of a single high-priority tenant, but also the more general use case where everyone is equal priority and we don't want a single bad actor to hurt things.
That said, another potential issue with MM op prioritization is that flush operations are responsible for keeping memory resources on the tablet server under control. I'm afraid that op prioritization may cause priority inversion: we may choose compactions of high-priority tablets ahead of memory-pressure-reducing flushes of low priority tablets. If the server is under memory pressure, maybe we should ignore priority and flush based on the normal memory-based prioritization?
The last question I have is about the use of gflags for this purpose. It seems like this should probably be a table property settable via the API at runtime, rather than a gflag. For example, we could use the 'extra config' functionality provided by https://gerrit.cloudera.org/c/12468/
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 26 Mar 2019 04:15:40 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#6).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit add a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allow administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 410 insertions(+), 155 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/6
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#2).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit add a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allow administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 402 insertions(+), 153 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/2
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
> Priority multipliers only applied to the compaction ops to improve performance.
> Priority multipliers could not applied to GC ops and flush ops, so they can work like before.
Oh okay; so it's just about prioritizing/deprioritizing compactions? That makes sense to me, and is a reasonable place to start.
Have you had a chance to consider Todd's last bit of feedback? I've copied it below:
> The last question I have is about the use of gflags for this purpose. It seems like this should probably be a table property settable via the API at runtime, rather than a gflag. For example, we could use the 'extra config' functionality provided by https://gerrit.cloudera.org/c/12468/
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Wed, 08 May 2019 05:27:08 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 7:
> Patch Set 6: Code-Review+2
>
> (2 comments)
>
> LGTM. Ideally I'd rather improve the maintenance manager so it accounts for the factors that make some tables high priority, e.g. by looking at how often tables are read or written. But this is fine for now.
>
> Can you file a JIRA with some details about what prompted this patch? What was the workload like on the high priority table? What was it like on the tables that were actually the target of MM ops? Some details might help plan future MM improvements.
I've file one here https://issues.apache.org/jira/browse/KUDU-2824
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 21 May 2019 10:59:23 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 6:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc
File src/kudu/util/maintenance_manager.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@467
PS5, Line 467: if (most_logs_retained_bytes_ram_anchored_op &&
> Could you add a comment to explain what we're looking for here?
Done
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@602
PS5, Line 602:
> You can CHECK() on this returning true; this command line option is known t
Done
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Sat, 18 May 2019 05:58:36 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
> 3. Like rocksDB, we keep two kind of thread pools, one for flush, and the other for compaction(and GC in Kudu).
How does this work? Are priority multipliers only applied to the compaction/GC pool?
Separately, if the goal is to do some sort of resource management on tablets, shouldn't we also consider prioritization at the point of ingest (i.e. during a Write RPC)? By only considering the MM we're potentially putting ourselves in difficult situations (e.g. a bunch of data can't be GC'ed because although it was written successfully, the tablet is "low" priority so the MM has deprioritized WAL GC for it).
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 07 May 2019 19:41:02 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 6:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc
File src/kudu/util/maintenance_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@345
PS5, Line 345: manager_->RegisterOp(&op);
> Likewise, does this increase the chance of the test failing, if the two rem
Done
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc
File src/kudu/util/maintenance_manager.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@342
PS5, Line 342: CHECK_OK(thread_pool_->SubmitFunc(boost::bind(&MaintenanceManager::LaunchOp, this, op)));
:
> You can combine this.
Done
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Sat, 18 May 2019 05:56:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 7: Verified+1
Failures look unrelated.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 21 May 2019 20:21:58 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 6: Code-Review+2
(2 comments)
LGTM. Ideally I'd rather improve the maintenance manager so it accounts for the factors that make some tables high priority, e.g. by looking at how often tables are read or written. But this is fine for now.
Can you file a JIRA with some details about what prompted this patch? What was the workload like on the high priority table? What was it like on the tables that were actually the target of MM ops? Some details might help plan future MM improvements.
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG@9
PS6, Line 9: add
adds
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG@14
PS6, Line 14: allow
allows
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Mon, 20 May 2019 17:22:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Add privilege maintenance thread pool for privilege tables and tablets
......................................................................
Patch Set 1:
> Patch Set 1:
>
> > Priority multipliers only applied to the compaction ops to improve performance.
> > Priority multipliers could not applied to GC ops and flush ops, so they can work like before.
>
> Oh okay; so it's just about prioritizing/deprioritizing compactions? That makes sense to me, and is a reasonable place to start.
>
Yes
> Have you had a chance to consider Todd's last bit of feedback? I've copied it below:
>
> > The last question I have is about the use of gflags for this purpose. It seems like this should probably be a table property settable via the API at runtime, rather than a gflag. For example, we could use the 'extra config' functionality provided by https://gerrit.cloudera.org/c/12468/
I think I have to wait this patch to be merged before adding a priority property into table. Before that, I could keep to use gflags, and we can controll server one by one by gflags.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Wed, 08 May 2019 06:10:22 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 5:
> Patch Set 2:
>
> (3 comments)
fixed
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Fri, 17 May 2019 02:15:14 +0000
Gerrit-HasComments: No
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 5:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/tablet/tablet_mm_ops.h
File src/kudu/tablet/tablet_mm_ops.h:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/tablet/tablet_mm_ops.h@62
PS5, Line 62: void UpdateStats(MaintenanceOpStats* stats) OVERRIDE;
If you're already cleaning this up, you could also convert 'OVERRIDE' into 'override', since C++11 supports it directly.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc
File src/kudu/util/maintenance_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@77
PS5, Line 77: void SetUp() OVERRIDE {
Nit: 'override' is more C++-11 compliant.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@253
PS5, Line 253: op1.set_sleep_time(MonoDelta::FromMilliseconds(10));
This doesn't seem look a good change: by going from a sleep time of 1000ms to 10ms, the chance of UnregisterOp on L263 happening during the sleep decreases. Why make this change?
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager-test.cc@345
PS5, Line 345: op.set_sleep_time(MonoDelta::FromMilliseconds(10));
Likewise, does this increase the chance of the test failing, if the two remaining runs happen before we call the ASSERT_EVENTUALLY below?
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h
File src/kudu/util/maintenance_manager.h:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h@233
PS5, Line 233: virtual const std::string& table_id() const = 0;
This is a different visibility than the other table_id definitions in this class hierarchy. Why?
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.h@311
PS5, Line 311: typedef std::unordered_map<std::string, int32_t> TablePriorities;
I assume the map keys are table names? That gets tricky when tables are renamed. Hoping the table extra config integration will address that.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc
File src/kudu/util/maintenance_manager.cc:
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@a531
PS5, Line 531:
:
:
I take it you dropped this because you didn't want to worry about locking table_priorities_? In your experience, is the best op not particularly useful?
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@100
PS5, Line 100: DEFINE_string(maintenance_manager_table_priorities, "",
: "Priorities of tables, semicolon-separated list of table-priority pairs, and each "
: "table-priority pair is combined by table id, colon and priority level. Priority "
: "level is ranged in [-FLAGS_max_priority_range, FLAGS_max_priority_range]");
: TAG_FLAG(maintenance_manager_table_priorities, advanced);
: TAG_FLAG(maintenance_manager_table_priorities, experimental);
: TAG_FLAG(maintenance_manager_table_priorities, runtime);
I'm not really going to review this aspect of your patch, because I assume this will move into the 'table extra config' when that's merged.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@342
PS5, Line 342: Status s = thread_pool_->SubmitFunc(boost::bind(&MaintenanceManager::LaunchOp, this, op));
: CHECK(s.ok());
You can combine this.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@467
PS5, Line 467: if (most_logs_retained_bytes_op &&
Could you add a comment to explain what we're looking for here?
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@484
PS5, Line 484: if (best_perf_improvement_op && best_perf_improvement > 0) {
Here too.
http://gerrit.cloudera.org:8080/#/c/12852/5/src/kudu/util/maintenance_manager.cc@602
PS5, Line 602: if (!google::GetCommandLineOption("maintenance_manager_table_priorities",
You can CHECK() on this returning true; this command line option is known to always exist.
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Fri, 17 May 2019 18:59:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit adds a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allows administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Reviewed-on: http://gerrit.cloudera.org:8080/12852
Tested-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Adar Dembo <ad...@cloudera.com>
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 410 insertions(+), 155 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/12852 )
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG@9
PS6, Line 9: add
> adds
Done
http://gerrit.cloudera.org:8080/#/c/12852/6//COMMIT_MSG@14
PS6, Line 14: allow
> allows
Done
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 21 May 2019 09:57:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] [maintenance] Support priorities for tables in MM compaction
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12852
to look at the new patch set (#4).
Change subject: [maintenance] Support priorities for tables in MM compaction
......................................................................
[maintenance] Support priorities for tables in MM compaction
This commit add a feature to specify different priorities for table compaction.
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's
maintenance OPs to be launched when their scores are not the highest, even if
the table the tablet belongs to is high priority for Kudu users. This patch
allow administators to specify different priorities for tables by gflags, these
maintenance OPs of these high priority tables have greater chance to be launched.
Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
---
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tablet/tablet_mm_ops-test.cc
M src/kudu/tablet/tablet_mm_ops.cc
M src/kudu/tablet/tablet_mm_ops.h
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tablet/tablet_replica_mm_ops.cc
M src/kudu/tablet/tablet_replica_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
M src/kudu/util/maintenance_manager.proto
13 files changed, 404 insertions(+), 153 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/12852/4
--
To view, visit http://gerrit.cloudera.org:8080/12852
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3ea3b73505157678a8fb551656123b64e6bfb304
Gerrit-Change-Number: 12852
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>