[jira] [Closed] (FLINK-31338) support infer parallelism for flink table store

["Jingsong Lee (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-31338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jingsong Lee closed FLINK-31338.
--------------------------------
    Resolution: Fixed

Just use github pr/issues.

> support  infer parallelism for flink table store
> ------------------------------------------------
>
>                 Key: FLINK-31338
>                 URL: https://issues.apache.org/jira/browse/FLINK-31338
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table Store
>    Affects Versions: table-store-0.3.0
>            Reporter: Jun Zhang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: table-store-0.4.0
>
>
> When using flink  to query the fts table, we can config the scan parallelism by set the scan.parallelism, but the user may do not know how much parallelism should be used,  setting a too large parallelism will cause resource waste, setting the parallelism too small will cause the query to be slow, so we can add parallelism infer.
> The function is enabled by default. the parallelism is equal to the number of read splits. Of course, the user can manually turn off the infer function. In order to prevent too many datafiles from causing excessive parallelism, we also set a max infer parallelism. When the infer parallelism exceeds the setting, use the max parallelism.
> In addition, we also need to compare with the limit in the select query statement to get a more appropriate parallelism in the case of limit pushdown, for example we have a sql select * from table limit 1, and finally we infer the parallelism is 10, but we only one parallel is needed , besause we only need one data .



--
This message was sent by Atlassian Jira
(v8.20.10#820010)