You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Yingchun Lai (JIRA)" <ji...@apache.org> on 2019/05/21 10:40:00 UTC
[jira] [Updated] (KUDU-2824) Make some tables in high priority in
MM compaction
[ https://issues.apache.org/jira/browse/KUDU-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yingchun Lai updated KUDU-2824:
-------------------------------
Description:
In a Kudu cluster with thousands of tables, it's hard for a specified tablet's maintenance OPs to be launched when their scores are not the highest, even if the table the tablet belongs to is high priority for Kudu users.
For example, table A has 10 tablets and has total size of 1G, table B has 1000 tablets and has total size of 100G. Both of them have similar update writes, i.e. DRSs have similar overlaps, similar redo/undo logs, so they have similar compaction scores. However, table A has much more reads than table B, but table A and B are equal in MM, their DRS compactions are lauched equally, we have to suffer a long time util most of tablets have been compacted in the cluster.
So, maybe we can introduce some algorithm to detect high priority tables and speed up compaction of these tables?
> Make some tables in high priority in MM compaction
> --------------------------------------------------
>
> Key: KUDU-2824
> URL: https://issues.apache.org/jira/browse/KUDU-2824
> Project: Kudu
> Issue Type: Improvement
> Components: tserver
> Affects Versions: 1.9.0
> Reporter: Yingchun Lai
> Assignee: Yingchun Lai
> Priority: Minor
> Labels: MM, compaction, maintenance, priority
>
> In a Kudu cluster with thousands of tables, it's hard for a specified tablet's maintenance OPs to be launched when their scores are not the highest, even if the table the tablet belongs to is high priority for Kudu users.
> For example, table A has 10 tablets and has total size of 1G, table B has 1000 tablets and has total size of 100G. Both of them have similar update writes, i.e. DRSs have similar overlaps, similar redo/undo logs, so they have similar compaction scores. However, table A has much more reads than table B, but table A and B are equal in MM, their DRS compactions are lauched equally, we have to suffer a long time util most of tablets have been compacted in the cluster.
> So, maybe we can introduce some algorithm to detect high priority tables and speed up compaction of these tables?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)