You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Jane Chan (Jira)" <ji...@apache.org> on 2022/05/06 07:35:00 UTC

[jira] [Updated] (FLINK-27526) Support scaling bucket number for FileStore

     [ https://issues.apache.org/jira/browse/FLINK-27526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jane Chan updated FLINK-27526:
------------------------------
    Description: 
Currently, TableStore does not support changing the number of the bucket (denoted by config option 'table-storage.bucket') once the managed table is created. The reason is that the LSM tree is built under bucket level, and thus writing with different bucket numbers will cause the same record to be hashed to another bucket and leads to data corruption. In the release-0.1, TableStore will detect this change and throw an exception when scanning. See FLINK-27316.

However, this is not flexible and user-friendly. To be more specific, if the bucket number remains unchanged, the number of files under each bucket will grow fast as time passes, slowing down the scan speed to restore the LSM tree and finally influencing read and write latency.

> Support scaling bucket number for FileStore
> -------------------------------------------
>
>                 Key: FLINK-27526
>                 URL: https://issues.apache.org/jira/browse/FLINK-27526
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table Store
>    Affects Versions: table-store-0.2.0
>            Reporter: Jane Chan
>            Assignee: Jane Chan
>            Priority: Major
>             Fix For: table-store-0.2.0
>
>
> Currently, TableStore does not support changing the number of the bucket (denoted by config option 'table-storage.bucket') once the managed table is created. The reason is that the LSM tree is built under bucket level, and thus writing with different bucket numbers will cause the same record to be hashed to another bucket and leads to data corruption. In the release-0.1, TableStore will detect this change and throw an exception when scanning. See FLINK-27316.
> However, this is not flexible and user-friendly. To be more specific, if the bucket number remains unchanged, the number of files under each bucket will grow fast as time passes, slowing down the scan speed to restore the LSM tree and finally influencing read and write latency.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)