[jira] [Created] (HUDI-3072) AutoCommit misses to detect write conflicts during concurrent transactions

["Manoj Govindassamy (Jira)" <ji...@apache.org>]

Manoj Govindassamy created HUDI-3072:
----------------------------------------

             Summary: AutoCommit misses to detect write conflicts during concurrent transactions
                 Key: HUDI-3072
                 URL: https://issues.apache.org/jira/browse/HUDI-3072
             Project: Apache Hudi
          Issue Type: Task
          Components: Writer Core
            Reporter: Manoj Govindassamy
            Assignee: Manoj Govindassamy
             Fix For: 0.11.0


When auto commit is enabled, the table write operation triggers the commit of the write result set. In this code path the transaction is started just in the autoCommit block and the meta client here is not the latest one and doesn't reflect the latest committed data set. The meta client it uses to detect the conflicts does't show the potential commits from the concurrent transactions and there by can miss out to detect the write conflicts during concurrent transactions. 

 

In the write and explicit commit code path, the table and meta client is built again inside the transaction so as to get the latest refreshed view of the timeline. It sees all the commits since the current inflight write started and able to detect the write conflicts reliably.

 

Need to improve the autoCommit code path to detect the write conflicts reliably. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)