You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2023/02/27 08:39:39 UTC
[GitHub] [iceberg] Fokko opened a new issue, #6945: Python: Inconsistency around timezones
Fokko opened a new issue, #6945:
URL: https://github.com/apache/iceberg/issues/6945
### Apache Iceberg version
main (development)
### Query engine
Other
### Please describe the bug 🐞
With PyIceberg when we filter a complete DataFile, we end up with:
```
ArrowInvalid: Schema at index 1 was different:
vendor_id: int32
pickup_time: timestamp[us, tz=+00:00]
pickup_location_id: int32
dropoff_time: timestamp[us, tz=+00:00]
dropoff_location_id: int32
passenger_count: int32
trip_distance: double
ratecode_id: int32
payment_type: int32
total_amount: double
fare_amount: double
tip_amount: double
tolls_amount: double
mta_tax: double
improvement_surcharge: double
congestion_surcharge: double
extra_surcharges: double
store_and_forward_flag: string
vs
vendor_id: int32
pickup_time: timestamp[us, tz=UTC]
pickup_location_id: int32
dropoff_time: timestamp[us, tz=UTC]
dropoff_location_id: int32
passenger_count: int32
trip_distance: double
ratecode_id: int32
payment_type: int32
total_amount: double
fare_amount: double
tip_amount: double
tolls_amount: double
mta_tax: double
improvement_surcharge: double
congestion_surcharge: double
extra_surcharges: double
store_and_forward_flag: string
```
We get a `+00:00` from the empty tables that we're `concat`'ing, and a `UTC` from the ones that actually contain data:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on issue #6945: Python: Inconsistency around timezones
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #6945:
URL: https://github.com/apache/iceberg/issues/6945#issuecomment-1445924812
Maybe we should also change the default timezone to UTC, since I think it is more common to have `UTC` as default, instead of `+00:00`:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko closed issue #6945: Python: Inconsistency around timezones
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #6945: Python: Inconsistency around timezones
URL: https://github.com/apache/iceberg/issues/6945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org