You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/13 07:16:29 UTC

[GitHub] [spark] sadikovi commented on pull request #33639: [SPARK-36645][SQL] Aggregate (Min/Max/Count) push down for Parquet

sadikovi commented on pull request #33639:
URL: https://github.com/apache/spark/pull/33639#issuecomment-941996560


   @huaxingao @sunchao Would you be able to elaborate on testing for this change? I am curious how the code handles some esoteric cases such as UINT32 and UINT64 min and max and UTF-8 strings, I remember Parquet stats had a bug regarding those some time ago. Also, I believe stats for strings can be truncated (https://github.com/apache/parquet-mr/blob/8ae7f31e36a298804435565e0cae584aac90f6d5/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetOutputFormat.java#L150), I am curious how this is handled or taken into account. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org