[jira] [Assigned] (KYLIN-6011) After the build task is completed, it is not cleared from the lock table, causing the remaining build tasks to fail to start

["Guoliang Sun (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KYLIN-6011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guoliang Sun reassigned KYLIN-6011:
-----------------------------------

    Assignee: Guoliang Sun

> After the build task is completed, it is not cleared from the lock table, causing the remaining build tasks to fail to start
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-6011
>                 URL: https://issues.apache.org/jira/browse/KYLIN-6011
>             Project: Kylin
>          Issue Type: Bug
>    Affects Versions: 5.0.0
>            Reporter: Guoliang Sun
>            Assignee: Guoliang Sun
>            Priority: Major
>
> h3. Root cause
> META and ROUTINE type tasks are set as the highest priority, see: `org.apache.kylin.job.execution.ExecutableManager#checkAndSubmitCronJob`.  
> The job lock table prioritizes checking and cleaning the earliest created records when clearing expired locks, and by default retrieves only 30 entries at a time (`kylin.job.master-pull-batch-size`).  
> If a large number of regular build tasks (> 30) are submitted first, followed by a large number of META or ROUTINE type tasks, the job lock table will fail to clean up completed META or ROUTINE tasks.  
> The job scheduler will then continuously read completed META or ROUTINE tasks from the job lock table, preventing new tasks from being scheduled.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)