You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "WillAyd (via GitHub)" <gi...@apache.org> on 2024/01/26 17:24:10 UTC
[I] UTC Timezone Equality is case sensitive [arrow]
WillAyd opened a new issue, #39809:
URL: https://github.com/apache/arrow/issues/39809
### Describe the bug, including details regarding any error messages, version, and platform.
Not sure if this is a bug but I found the behavior surprising
```
>>> import pyarrow as pa
>>> pa.timestamp("us", "utc") == pa.timestamp("us", "utc")
True
>>> pa.timestamp("us", "utc") == pa.timestamp("us", "UTC")
False
```
I suppose it is doing a very literal string comparison on the timezone
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] UTC Timezone Equality is case sensitive [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on issue #39809:
URL: https://github.com/apache/arrow/issues/39809#issuecomment-1914681651
The comparison of DataTypes is done in Arrow C++ and for timestamps it is comparing the exact timezone string (it is saved as a string) as you have mentioned.
To do a logical comparison of timezones one should probably use a dedicated library for that. In Python you could try with:
```python
>>> import pytz
>>> pytz.timezone(pa.timestamp("us", "UTC").tz)
<UTC>
>>> pytz.timezone(pa.timestamp("us", "utc").tz)
<UTC>
>>> pytz.timezone(pa.timestamp("us", "utc").tz) == pytz.timezone(pa.timestamp("us", "UTC").tz)
```
and then separately compare the timestamp unit:
```python
>>> pa.timestamp("us", "utc").unit == pa.timestamp("us", "UTC").unit
True
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] UTC Timezone Equality is case sensitive [arrow]
Posted by "WillAyd (via GitHub)" <gi...@apache.org>.
WillAyd commented on issue #39809:
URL: https://github.com/apache/arrow/issues/39809#issuecomment-1914984014
Thanks for the input @AlenkaF . Another option I would propose is to disallow "utc" and force "UTC". The former is technically not part of the Timezone Database List
For instance, if you try to do the same thing in polars you get an error. Taking a slightly modified example from their docs
```python
>>> import polars as pl
>>> ts = ["2021-03-27 03:00", "2021-03-28 03:00"]
>>> tz_naive = pl.Series("tz_naive", ts).str.to_datetime()
>>> tz_aware = tz_naive.dt.replace_time_zone("utc").rename("tz_aware")
```
yields:
```python
ComputeError: unable to parse time zone: 'utc'. Please check the Time Zone Database for a list of available time zones
```
I came across this issue when trying to exchange arrow data across the different libraries
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] UTC Timezone Equality is case sensitive [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39809:
URL: https://github.com/apache/arrow/issues/39809#issuecomment-1916309229
We actually also raise an error if you try to _use_ the lower case "utc", exactly for that reason of not being part of the database. For example:
```
In [18]: arr_naive = pa.array([pd.Timestamp("2021-01-01 09:00")])
In [19]: pc.assume_timezone(arr_naive, "utc")
...
ArrowInvalid: Cannot locate timezone 'utc': utc not found in timezone database
```
However, the issue here is that we don't do any validation of the timezone at all at construction time of the type (or when reading in a file, or ..). At the moment you can perfectly write complete nonsense like `pa.timestamp("us", tz="blabla")`.
I assume one reason for this is because there is not necessarily a guarantee that a timezone database is available to perform this check (i.e. typically on windows).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org