You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rotem-ad (via GitHub)" <gi...@apache.org> on 2025/01/01 09:03:56 UTC
Re: [I] Support for timestamp downcasting when loading data to iceberg tables [iceberg-python]
rotem-ad commented on issue #1045:
URL: https://github.com/apache/iceberg-python/issues/1045#issuecomment-2566918400
I've faced the same issue when loading data using [Table.add_files](https://py.iceberg.apache.org/reference/pyiceberg/table/#pyiceberg.table.Table.add_files) method. It fails and shows this error:
```
TypeError: Iceberg does not yet support 'ns' timestamp precision. Use 'downcast-ns-timestamp-to-us-on-write' configuration property to automatically downcast 'ns' to 'us' on write.
```
As @fusion2222 already mentioned, setting `PYICEBERG_DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE` to `'true'` does not help. It's completely ignored since its value is not passed to `_check_pyarrow_schema_compatible` function when using `add_files`.
IMO this is a pretty **serious bug**, but **the fix is super easy** (as shown in @fusion2222 patch).
Can someone please merge the fix for this? If not I guess I'll try to create a PR for the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org