[GitHub] [iceberg] pvary commented on issue #2301: Lock remains in HMS if HiveTableOperations gets killed (direct process shutdown - no signals) after lock is acquired

[GitBox <gi...@apache.org>]

pvary commented on issue #2301:
URL: https://github.com/apache/iceberg/issues/2301#issuecomment-1341362767

   > We faced similar issues:
   > 
   > 1. We are using Flink and some processes attempt to modify a table concurrently, e.g.: ingestion process and data compaction process. If one of them fails then a lock can remain in `ACQUIRED` state. We clear all the logs from another process in `onInit` function.
   
   This is problematic. You should not `clear` locks. You should only do this when you are sure that the other process is not writing/committing.
   
   > 2. This one is weird. We see a lot of errors in logs: `Base metadata location '%s' is not same as the current table metadata location '%s' for %s.%s`  [see](https://github.com/apache/iceberg/blob/6b8f7e0e31a81029b478e7757aba749f5ed27f42/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L295), which means table has been updated regardless of an acquired lock.
   
   This is exactly the case which could happen when the locks are removed during a commit and another process is trying to commit concurrenly.
   
   The correct solution for the issue is to try to make sure that the lock is removed by the failing process
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org