[GitHub] [iceberg] kbendick commented on pull request #2552: Core: Fix the NPE while updating event in the context of eventual consistency.

[GitBox <gi...@apache.org>]

kbendick commented on pull request #2552:
URL: https://github.com/apache/iceberg/pull/2552#issuecomment-833238321


   > I think this is a good fix to the NPE. But I do want to make sure we warn folks that an eventually consistent Catalog can break the history of an Iceberg table. This would only be acceptable for a catalog which guaranteed consistent reads during the lock phase.
   > 
   > My thinking is basically
   > 
   > ```
   > Consistent - Read Your Writes  Catalog = Best
   > Eventually Consistent Reads - Consistent when locking/committing = Adequate but not great
   > Eventually consistent during lock/commit = Will cause lost writes, non-linearizable histories and weirdness. 
   > ```
   
   Is this something that could come up with the DynamoDB based lock? I agree this is a good fix to the NPE, but I have personally had weird edge cases with other software that uses a DynamoDB table as a lock (such as terraform). Ideally testing handles this and the DynamoDB / Glue catalog doesn't suffer from these issues. Possibly I need to review the DynamoDB table lock again, but felt it might be worth noting as I've had weird issues with terraform table locks and consistency issues.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org