[GitHub] [hive] kuczoram commented on pull request #1507: HIVE-24163: Dynamic Partitioning Insert fail for MM table fail during…

[GitBox <gi...@apache.org>]

kuczoram commented on pull request #1507:
URL: https://github.com/apache/hive/pull/1507#issuecomment-696209500


   The file listing in the Utilities.getFullDPSpecs method was not correct for MM tables and for ACID tables when direct insert was on. This method returned all partitions from these tables, not just the ones affected by the current query. Because of this, the lineage information for inserting with dynamic partitioning into tables like these was not correct. Compared the lineage information with when inserting into external tables and for external tables only the partitions are present which are affected by the query. This is because for external tables when inserting into the table, the data first get written into the staging dir and when listing the partitions, this directory is checked and it contains only the newly inserted data. But for MM tables and ACID direct insert, the staging dir is missing, so it will check the table directory and lists everything from it.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org