[GitHub] [iceberg] ahshahid commented on issue #6039: Spark : Perf enhancement by leveraging Dynamic Partition Pruning rule of spark for non partition columns used as join condition

[GitBox <gi...@apache.org>]

ahshahid commented on issue #6039:
URL: https://github.com/apache/iceberg/issues/6039#issuecomment-1301385359

   I am still working on perf aspect.. In a limited tpcds test , it turns out that dpp query is way expensive.. But interestingly I noticed, that though the manifest file contains bounds information for non partition column, they are not being read / used ( may be bcox tableContext has boolean colStats as hardcoded false.. still going through the code.
   I want to check the impact on perf, if these stats are available at manifest level for non partitioning col..


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org