You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/12/02 13:51:00 UTC
[GitHub] [iceberg] marton-bod opened a new pull request #3652: Hive: HiveCatalog should remove HMS stats for certain engines based on config
marton-bod opened a new pull request #3652:
URL: https://github.com/apache/iceberg/pull/3652
The Hive engine keeps the HMS stats updated after each write to a table, provided that hive.stats.autogather is turned on. If other engines are interacting with the same table however, e.g. Spark writing to the same table, they might not update the HMS stats but also not invalidate the stats within HMS either, leaving it stale but HMS believing that it's up-to-date. This PR introduces a config which can enable/disable HMS stats invalidation in HiveCatalog for each engine.
cc @pvary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #3652: Hive: HiveCatalog should remove HMS stats for certain engines based on config
Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #3652:
URL: https://github.com/apache/iceberg/pull/3652#issuecomment-986613079
Thanks for the patch @marton-bod!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary commented on pull request #3652: Hive: HiveCatalog should remove HMS stats for certain engines based on config
Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #3652:
URL: https://github.com/apache/iceberg/pull/3652#issuecomment-984659334
The Hive part looks good to me, but not sure why the Spark test failing. Could you please check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pvary merged pull request #3652: Hive: HiveCatalog should remove HMS stats for certain engines based on config
Posted by GitBox <gi...@apache.org>.
pvary merged pull request #3652:
URL: https://github.com/apache/iceberg/pull/3652
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org