You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "sharkdtu (via GitHub)" <gi...@apache.org> on 2025/02/28 12:51:50 UTC
[I] [Python] Timestamptz type loses its time zone after map transforming. [arrow]
sharkdtu opened a new issue, #45644:
URL: https://github.com/apache/arrow/issues/45644
### Describe the bug, including details regarding any error messages, version, and platform.
Timestamptz type loses its time zone after map transforming.
## Versions / Dependencies
```
In [44]: pa.__version__
Out[44]: '19.0.1'
```
## Reproduction script
```
In [39]: import pyarrow as pa
In [40]: x = pa.array([1735689600, 1735689600, 1735689600], type=pa.timestamp("s", tz='UTC'))
In [41]: print(x.type)
timestamp[s, tz=UTC]
In [42]: y = x.to_numpy()
In [43]: print(y.dtype)
datetime64[s]
```
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF closed issue #45644: [Doc][Python] Timestamp with tz loses its time zone after `to_numpy`
URL: https://github.com/apache/arrow/issues/45644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]
Posted by "alex-anast (via GitHub)" <gi...@apache.org>.
alex-anast commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-4299938575
Hi @AlenkaF I've opened up a PR for this. Could you please take a look? š
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-2693674211
This behaviour is expected as NumPy datetimes are not timezone aware, see https://numpy.org/devdocs/reference/arrays.datetime.html#datetimes-and-timedeltas.
You can convert pyarrow tz-aware timestamp array to
- numpy datetime64 with loss of tz information,
- pandas tz-aware datetime64 dtype
- a Pyhon object
```python
arr_tz = pa.array([1735689600, 1735689600, 1735689600], type=pa.timestamp("s", tz='UTC'))
# numpy datetime64 dtype (losing tz information)
>>> arr_tz.to_numpy()
array(['2025-01-01T00:00:00', '2025-01-01T00:00:00',
'2025-01-01T00:00:00'], dtype='datetime64[s]')
```
```python
# pandas tz-aware datetime64 dtype
>>> arr_tz.to_pandas().array
0 2025-01-01 00:00:00+00:00
1 2025-01-01 00:00:00+00:00
2 2025-01-01 00:00:00+00:00
dtype: datetime64[s, UTC]
```
```python
# python object
>>> arr_tz.to_pandas(timestamp_as_object=True).to_numpy()
array([datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>),
datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>),
datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>)], dtype=object)
```
I will keep this issue open as this needs to be documented in https://arrow.apache.org/docs/python/numpy.html.
Also connected and need to go into the docs: https://github.com/apache/arrow/issues/41162.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]
Posted by "Bhupendra-glitch (via GitHub)" <gi...@apache.org>.
Bhupendra-glitch commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-3703303498
Hello @AlenkaF
Iād like to work on documenting the timezone behavior when converting PyArrow timestamp arrays to NumPy.
I plan to add a section to the Python NumPy documentation explaining the loss of timezone info and the recommended alternatives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org