[GitHub] [incubator-iceberg] rdblue commented on issue #732: [DISCUSS] Update metadata serially

[GitBox <gi...@apache.org>]

rdblue commented on issue #732: [DISCUSS] Update metadata serially
URL: https://github.com/apache/incubator-iceberg/issues/732#issuecomment-574446855
 
 
   Commits don't wait in a queue. Committers write out metadata optimistically and will retry if another process commits first. The location swap must be done to get a linear history, but we try to make the remainder of the commit a light-weight operation by caching changes are reusing them.
   
   For example, an append will create a new manifest file. When an append retries, it doesn't rewrite the manifest file from the first commit attempt. Instead, it updates the now-current metadata by adding the same manifest from the first try. That saves time and effectively does the majority of the metadata write work in parallel.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org