[jira] [Created] (HUDI-3692) Write cannot see inflight compaction when using metadata table

["Yuwei Xiao (Jira)" <ji...@apache.org>]

Yuwei Xiao created HUDI-3692:
--------------------------------

             Summary: Write cannot see inflight compaction when using metadata table
                 Key: HUDI-3692
                 URL: https://issues.apache.org/jira/browse/HUDI-3692
             Project: Apache Hudi
          Issue Type: Bug
          Components: metadata, writer-core
            Reporter: Yuwei Xiao
         Attachments: image-2022-03-23-17-52-38-672.png

The construction of metadata table accepts a timeline without compaction. So without further `sync()` called on the view, async in-flight compactions is not recognized by writers, leading to lost of updates.

!image-2022-03-23-17-52-38-672.png!

To reproduce:

1. insert 10 records to create file group A
2. update 5 records in file group A, produce log file 1 (make sure it is producing log file rather than re-write file group)
3. schedule compaction (do not execute)
4. delete 5 records in file group A, produce log file 2
5. insert the deleted records in file group A, produce log file 3
6. Read table, and the total number is 5 rather than the 10 records.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)