[GitHub] [hudi] prashantwason commented on pull request #3427: [HUDI-1292] Created a config to enable/disable syncing of metadata table.

[GitBox <gi...@apache.org>]

prashantwason commented on pull request #3427:
URL: https://github.com/apache/hudi/pull/3427#issuecomment-897222618


   To summarize:
   
   1. Async [clean, compact, cluster] - these are in the same process but run parallel in thread
       - We can take a lock in syncMetadata()
       - We can sync only as part of commit
       - We can depend on transaction support
   2. Parallel [clean, compact, cluster] - these are in different processes
       - We can use the proposed setting to disable sync in all but one pipeline
       - We can depend on transaction support
   
   Only transaction support fixed both the cases but this may not come by this weekend ([I have a patch for you to review](https://github.com/apache/hudi/commit/f417390aed56c05dbaedeac6ecf47294996bb591) though). In the short run, we need to:
   1. Enable the setting in this PR
   2. Handle the async operations as @vinothchandar is proposing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org