[GitHub] [iceberg] RussellSpitzer commented on pull request #4463: Docs: update Spark Write doc for partitioned tables

[GitBox <gi...@apache.org>]

RussellSpitzer commented on PR #4463:
URL: https://github.com/apache/iceberg/pull/4463#issuecomment-1103864270

   I think there is a misconception here between data layout sort orders and the ordering of the data in a query. The orders documented by iceberg dictate how the data is physically stored in files on the disc and happens based on distribution and ordering requirements in the DSV2 api. The functions mentioned here and their sql only effect Spark on read and cause shuffles to order output for the reader but do not effect the on disc layout.
   
   The file based ordering is to make pruning of files during scan planning more efficient while the ordering during read is to satisfy user query requirements.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org