[GitHub] [iceberg] rdblue commented on pull request #1975: Core: add sort order id to content file

[GitBox <gi...@apache.org>]

rdblue commented on pull request #1975:
URL: https://github.com/apache/iceberg/pull/1975#issuecomment-751829634


   > Not sure if sort order should be nullable by default or 0 (from unsorted_order)
   
   The field should be optional because v1 manifests will not have the order field. Iceberg will read the value as null, so I think it makes sense to use null. And you're right about not storing it for position deletes.
   
   > Do we want only sort order id, or actual sort order struct?
   
   We want the ID. Sort orders are attached to table metadata, so loading the order should be a simple hash map lookup.
   
   > For the next PR, do we assume the table's current sort order id is the authoritative place to get sort order information when adding a new file?
   
   No. Engines must specify which sort order was used to write a file explicitly. So this needs to be exposed in the DataFile and DeleteFile builders. By default, we should write either null or 0 (unordered). Probably null.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org