[GitHub] [iceberg] raptond commented on issue #2540: Hive: Lock Issues with multithreaded commits

[GitBox <gi...@apache.org>]

raptond commented on issue #2540:
URL: https://github.com/apache/iceberg/issues/2540#issuecomment-828777965


   The problem we face internally with several customers is, after a while, the Iceberg table cannot be added any more data. When I say "any more data", I mean, data files are created, snapshot is created but the commit can never be successful. At this point, a manual intervention of clearing the locks for the given iceberg table is required.
   
   Some cases that we investigated had 
   1. several jobs 
   2. and within each job, several threads
   
   trying to commit to the same Iceberg table. 
   
   > We should only be blocking threads which would like to commit to the same table simultaneously, not to any Iceberg table in general (since the HMS locks are also on the table level).
   👍 to this. 
   
   >  we release the HMS lock, in this case the commits will happen in a Round Robin fashion between the JVMs.
   👍 to this too.
   
   The `share-the-hms-lock-within-JVM-for-a-single-iceberg-table` proposed by @RussellSpitzer would save the situation (2) very effectively. IMHO with additional considerations like, [a] release the HMS lock as soon as the last thread returned the lock back and [b] releasing it by time for fair usage even if more threads are in the queue waiting for the lock. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org