[jira] [Assigned] (KYLIN-4806) Native kubernetes support for kylin streaming receivers

["Xiaoxiang Yu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KYLIN-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiaoxiang Yu reassigned KYLIN-4806:
-----------------------------------

    Assignee: Xingxing Di

> Native kubernetes support for kylin streaming receivers
> -------------------------------------------------------
>
>                 Key: KYLIN-4806
>                 URL: https://issues.apache.org/jira/browse/KYLIN-4806
>             Project: Kylin
>          Issue Type: New Feature
>          Components: Real-time Streaming
>            Reporter: Xingxing Di
>            Assignee: Xingxing Di
>            Priority: Major
>
> Currently kylin manually maintains stream receiver replicasets. Under this design, when we have many cubes, we cannot achieve a perfect balance between resource isolation and maintenance costs:
>  1. Maintain a large number of replicasets to maintain resource isolation, which will cause a lot of maintenance costs
> 2. Maintain less replicasets to reduce maintenance costs, so one replicaset should be used for multiple cubes, resource isolation will be a problem. eg: One cube’s source data(Kafka topic) has grown abnormally, which may cause the streaming receiver process unstable.
> In the context of cloud native, kylin should add native support for kubernetes, and then we can easily achieve *cube level streaming receiver cluster* (a set of replicatset ) by define the necessary parameters such as parallelism, cpu, memory, etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)