You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "KnightChess (Jira)" <ji...@apache.org> on 2023/12/26 11:51:00 UTC

[jira] [Created] (HUDI-7259) Multiple tasks get the lock at the same time when use occ

KnightChess created HUDI-7259:
---------------------------------

             Summary: Multiple tasks get the lock at the same time when use occ
                 Key: HUDI-7259
                 URL: https://issues.apache.org/jira/browse/HUDI-7259
             Project: Apache Hudi
          Issue Type: Bug
          Components: multi-writer
            Reporter: KnightChess


config:
 * occ open
 * use FileSystemBasedLock
 * mdt is open in write defualt

there has three job, jobA, jobB, jobC, these three jobs are running at the same time.

jobA get lock success, jobB has been trying to get lock, jobC also try to get lock.

jobB failed because can not get lock, but it delete lock file when close write client, now, jobC will get lock, it cause concurrent problem.

In our case, jobC will rollback jobA mdt commit which has been succeed commited. So, the data table timeline has the repleaseCommit instance, but mdt without this update, it cause partition path be deleted and can not reserve the latest file split in our case

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)