[jira] [Updated] (HUDI-5303) Allow users to control the concurrency to submit jobs in clustering

["Hui An (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hui An updated HUDI-5303:
-------------------------
    Description: If there are sufficient resources in the clustering job, some clustering groups sometimes could still waits to be triggered, we use forkJoinPool to submit these jobs, and it's also difficult for clients to adjust this configure(--conf spark.driver.extraJavaOptions=-Djava.util.concurrent.ForkJoinPool.common.parallelism), and it could also affect other tasks using the forkJoinPool, so instead, we introduce a new threadPool to control the submitting job parallelism for the clustering.

> Allow users to control the concurrency to submit jobs in clustering
> -------------------------------------------------------------------
>
>                 Key: HUDI-5303
>                 URL: https://issues.apache.org/jira/browse/HUDI-5303
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: clustering, spark
>            Reporter: Hui An
>            Assignee: Hui An
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.13.0
>
>
> If there are sufficient resources in the clustering job, some clustering groups sometimes could still waits to be triggered, we use forkJoinPool to submit these jobs, and it's also difficult for clients to adjust this configure(--conf spark.driver.extraJavaOptions=-Djava.util.concurrent.ForkJoinPool.common.parallelism), and it could also affect other tasks using the forkJoinPool, so instead, we introduce a new threadPool to control the submitting job parallelism for the clustering.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)