You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Hong Shen (JIRA)" <ji...@apache.org> on 2018/10/15 13:54:00 UTC
[jira] [Comment Edited] (KUDU-2604) Add label for tserver
[ https://issues.apache.org/jira/browse/KUDU-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650188#comment-16650188 ]
Hong Shen edited comment on KUDU-2604 at 10/15/18 1:53 PM:
-----------------------------------------------------------
[Will Berkeley|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wdberkeley] Thanks a lot for your comment.
Our initial idea was to first consider the same tservers as the table label when choosing a replica, and then select tserver based on Location Awareness in these tservers.
But maybe considering the structure's label would be better, such as the label (datacenter, rack location, machine type) with three attributes. Each tserver has these properties. Then we can implement configurable policies. For example, one of the policies is to first select the corresponding tservers according to the machine type specified by the table, then select tservers based on Location Awareness in these tservers. Of course, we can also implement more policies if needed. As shown below.
!image-2018-10-15-21-52-21-426.png|width=647,height=329!
I have open up the design doc's permissions.
was (Author: shenhong):
[Will Berkeley|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wdberkeley] Thanks a lot for your comment.
Our initial idea was to first consider the same tservers as the table label when choosing a replica, and then select tserver based on Location Awareness in these tservers.
But maybe considering the structure's label would be better, such as the label (datacenter, rack location, part) with three attributes. Each tserver has these properties. Then we can implement configurable policies. For example, one of the policies is to first select the corresponding tservers according to the label specified by the table, then select tservers based on Location Awareness in these tservers. Of course, we can also implement more policies if needed. As shown below.
!屏幕快照 2018-10-15 21.09.35.png|width=589,height=292!
I have open up the design doc's permissions.
> Add label for tserver
> ---------------------
>
> Key: KUDU-2604
> URL: https://issues.apache.org/jira/browse/KUDU-2604
> Project: Kudu
> Issue Type: New Feature
> Reporter: Hong Shen
> Priority: Major
> Attachments: image-2018-10-15-21-52-21-426.png, 屏幕快照 2018-10-15 21.09.35.png
>
>
> When the cluster is bigger and bigger, big table with a lot of tablets will be distributed in almost all the tservers, when client write batch to the big table, it may cache connections to lots of tservers, the scalability may constrained.
> If the tablets in one table or partition only in a part of tservers, client will only have to cache connections to the part's tservers. So we propose to add label to tservers, each tserver belongs to a unique label. Client specified label when create table or add partition, the tablets will only be created on the tservers in specified label, if not specified, defalut label will be used.
> It will also benefit for:
> 1 Tserver across data center.
> 2 Heterogeneous tserver, like different disk, cpu or memory.
> 3 Physical isolating, especially IO, isolate some tables with others.
> 4 Gated Launch, upgrade tservers one by one label.
> In our product cluster, we have encounter the above issues and need to be resolved.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)