You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/12/11 04:06:49 UTC
[GitHub] [iceberg] jfz opened a new pull request #3717: Commit state should be unknown even if current location is not in history
jfz opened a new pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717
Currently when there's an exception during commit, we refresh table and check commit state by looking at if the new metadata location matches current or in history, and commit state is Failure if not, and the current metadata file is deleted.
There's a possibility that when the check happens, the metadata remote server (e.g. Hive metastore) may still be processing the commit and eventually succeed later, in this case, the commit succeeds but metadata is deleted, and the table is in bad state.
To address this, the commit state should be considered as unknown and propagated to user.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3717: Core: Commit state should be unknown even when new location is not in history
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717#issuecomment-991948588
I think this is correct. Another case is where the initial commit times out, but it succeeds on the server side, then another commit is successful. The metadata location would not match, but the commit was valid.
@RussellSpitzer, can you take a look at this as well? Good to get more than just one person looking at these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3717: Core: Commit state should be unknown even when new location is not in history
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717#issuecomment-991948630
Thanks for catching this, @jfz!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3717: Core: Commit state should be unknown even when new location is not in history
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717#issuecomment-991956860
Thanks, @jfz! And thanks for the additional review, @RussellSpitzer!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #3717: Core: Commit state should be unknown even when new location is not in history
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jfz commented on pull request #3717: Core: Commit state should be unknown even when new location is not in history
Posted by GitBox <gi...@apache.org>.
jfz commented on pull request #3717:
URL: https://github.com/apache/iceberg/pull/3717#issuecomment-992001873
Thanks @rdblue and @RussellSpitzer for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org