[GitHub] [arrow-rs] alamb opened a new issue, #2033: Parquet Writer Ignores "max statistics size" specification in `WriterProperties`

[GitBox <gi...@apache.org>]

alamb opened a new issue, #2033:
URL: https://github.com/apache/arrow-rs/issues/2033

   **Describe the bug**
   
   `WriterProperties::max_statistics_size` is ignored in the writer
   
   https://docs.rs/parquet/17.0.0/parquet/file/properties/struct.WriterProperties.html#method.max_statistics_size
   
   **To Reproduce**
   Set the stats size to `1` (byte) and notice that statistics are still happily created
   
   **Expected behavior**
   The statistics size limit should be respected (and documented more carefully -- like is it the total size of all statistics? If the limit is exceeded will partial statistics have been written, etc). Looking at the java or C++ parquet writer for inspiration is likely a good idea. 
   
   **Additional context**
   
   In https://github.com/apache/arrow-rs/pull/2022 @tustvold  fixed the writer to respect the "do/don't compute stats", however the "max size of computed statistics" setting is still ignored
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org