Re: [I] Support Defining PartitionSpec and SortOrder without field-ids in create_table [iceberg-python]

["Samreay (via GitHub)" <gi...@apache.org>]

Samreay commented on issue #338:
URL: https://github.com/apache/iceberg-python/issues/338#issuecomment-2456444991

   Hey @sungwy, just thought I'd chase this as well. The PR you linked is merged and 0.7.1 is now out, so does that mean there is a new way of specifying sort order we can use with pyarrow schemas? I've been trying to do it the way recommended by the doco and still running into `ValueError`
   
   ```python
   import polars as pl
   from pyiceberg.catalog.glue import GlueCatalog
   from pyiceberg.table.sorting import SortField, SortOrder
   from pyiceberg.transforms import IdentityTransform
   
   # Using the arrow schema as per https://py.iceberg.apache.org/#write-a-pyarrow-dataframe
   df = pl.DataFrame({"a": [1, 2, 3, 4, 5], "b": [5, 4, 3, 2, 1]}).to_arrow()
   
   glue_catalog = GlueCatalog(name="", properties={"write.parquet.compression-codec": "snappy"})
   table = glue_catalog.create_table(
       identifier="dev-cleaned.tmp_iceberg",
       schema=df.schema,
       location="s3:///mybucket/tmp",
       sort_order=SortOrder(SortField(source_id=1, transform=IdentityTransform())),
   )
   ```
   
   Gives:
   
   ```
   Exception has occurred: ValueError
   Could not find in old schema: 1 ASC NULLS FIRST
     File "/home/sam/arenko/flows-datalake/tmp_arrow.py", line 10, in <module>
       table = glue_catalog.create_table(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
   ValueError: Could not find in old schema: 1 ASC NULLS FIRST
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org