[GitHub] [incubator-iceberg] rdblue opened a new pull request #990: Fix Hadoop commit race condition

[GitBox <gi...@apache.org>]

rdblue opened a new pull request #990:
URL: https://github.com/apache/incubator-iceberg/pull/990


   This fixes #970. The problem happens when a table is updated concurrently in two processes and one process has multiple threads accessing the same `Table` instance. The multi-threaded process starts to commit and validates that its base `TableMetadata` is correct (e.g., version 8), the other process completes a commit to the table (version 9). In the process that is still committing, another thread refreshes the table causing the `version` to be updated (version 9). When committer writes its new `TableMetadata`, it will commit the wrong version (version 10) and the commit will succeed.
   
   The solution here is to ensure that version and metadata are accessed together using synchronized private methods, so there is no way to get metadata that doesn't correspond to the version, and so there is no way to see a version update but not a metadata update.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org