[GitHub] [iceberg] gaborkaszab commented on issue #6257: Partitions metadata table shows old partitions

[GitBox <gi...@apache.org>]

gaborkaszab commented on issue #6257:
URL: https://github.com/apache/iceberg/issues/6257#issuecomment-1396924742

   The "old" partitions are present in the output of a partition metadata table query when the table has merge-on-read for updates. This is because for the partition metadata table the delete files are not applied to the data files to see what partitions are remaining. I think this is a bit misleading because now querying the partitions metadata table can have different results depending on copy-on-write or merge-on-read was used.
   
   In order to fix this the delete files should be applied but that would break the idea of metadata table queries being lightweight and only requiring table metadata to produce the output. Regardless, I still think fixing this by applying the delete files would make sense (behind a flag?).
   Could you please share your thoughts, @szehon-ho @rdblue @danielcweeks ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org