You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/01/25 15:40:03 UTC
[GitHub] [iceberg] marton-bod opened a new pull request #2150: Hive: Avoid closing null writer in output committer abortTask
marton-bod opened a new pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150
In case we experience a failure during task commit (`HiveIcebergOutputCommitter#commitTask`), we'll have a problem during task abortion (`HiveIcebergOutputCommitter#abortTask`) because the committer has already removed its writer from the `HiveIcebergRecordWriter`'s static map, therefore `abortTask` will run into NPE when dereferencing the writer.
To solve this, the `commitTask` should call Map#get instead of Map#remove initially, and only remove the reference from the map once it has successfully completed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150#issuecomment-772104841
Thanks, @marton-bod! And thanks for reviewing, @pvary!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] marton-bod commented on pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
marton-bod commented on pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150#issuecomment-772322827
Thanks for the merge, @rdblue, and @pvary for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150#issuecomment-766925623
Can we have tests for this? (Like Mocked writer which throws an exception on close())
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150#issuecomment-766925623
Can we have tests for this? (Like Mocked writer which throws an exception on close())
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2150: Hive: Avoid closing null writer in output committer abortTask
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2150:
URL: https://github.com/apache/iceberg/pull/2150#issuecomment-772104841
Thanks, @marton-bod! And thanks for reviewing, @pvary!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org