You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "sharkdtu (via GitHub)" <gi...@apache.org> on 2025/02/28 12:51:50 UTC

[I] [Python] Timestamptz type loses its time zone after map transforming. [arrow]

sharkdtu opened a new issue, #45644:
URL: https://github.com/apache/arrow/issues/45644

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   Timestamptz type loses its time zone after map transforming.
   
   ## Versions / Dependencies
   ```
   In [44]: pa.__version__
   Out[44]: '19.0.1'
   ```
   
   ## Reproduction script
   
   ```
   In [39]: import pyarrow as pa
   
   In [40]: x = pa.array([1735689600, 1735689600, 1735689600], type=pa.timestamp("s", tz='UTC'))
   
   In [41]: print(x.type)
   timestamp[s, tz=UTC]
   
   In [42]: y = x.to_numpy()
   
   In [43]: print(y.dtype)
   datetime64[s]
   ```
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]

Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF closed issue #45644: [Doc][Python] Timestamp with tz loses its time zone after `to_numpy`
URL: https://github.com/apache/arrow/issues/45644


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]

Posted by "alex-anast (via GitHub)" <gi...@apache.org>.
alex-anast commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-4299938575

   Hi @AlenkaF I've opened up a PR for this. Could you please take a look? šŸ™ 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]

Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-2693674211

   This behaviour is expected as NumPy datetimes are not timezone aware, see https://numpy.org/devdocs/reference/arrays.datetime.html#datetimes-and-timedeltas.
   
   You can convert pyarrow tz-aware timestamp array to
   - numpy datetime64 with loss of tz information, 
   - pandas tz-aware datetime64  dtype
   - a Pyhon object
   
   ```python
   arr_tz = pa.array([1735689600, 1735689600, 1735689600], type=pa.timestamp("s", tz='UTC'))
   
   # numpy datetime64 dtype (losing tz information)
   >>> arr_tz.to_numpy()
   array(['2025-01-01T00:00:00', '2025-01-01T00:00:00',
          '2025-01-01T00:00:00'], dtype='datetime64[s]')
   ```
   ```python
   # pandas tz-aware datetime64 dtype
   >>> arr_tz.to_pandas().array
   0   2025-01-01 00:00:00+00:00
   1   2025-01-01 00:00:00+00:00
   2   2025-01-01 00:00:00+00:00
   dtype: datetime64[s, UTC]
   ```
   ```python
   # python object
   >>> arr_tz.to_pandas(timestamp_as_object=True).to_numpy()
   array([datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>),
          datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>),
          datetime.datetime(2025, 1, 1, 0, 0, tzinfo=<UTC>)], dtype=object)
   ```
   
   I will keep this issue open as this needs to be documented in https://arrow.apache.org/docs/python/numpy.html.
   Also connected and need to go into the docs: https://github.com/apache/arrow/issues/41162.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Doc][Python] Timestamp with tz loses its time zone after `to_numpy` [arrow]

Posted by "Bhupendra-glitch (via GitHub)" <gi...@apache.org>.
Bhupendra-glitch commented on issue #45644:
URL: https://github.com/apache/arrow/issues/45644#issuecomment-3703303498

   Hello @AlenkaF 
    I’d like to work on documenting the timezone behavior when converting PyArrow timestamp arrays to NumPy.
   I plan to add a section to the Python NumPy documentation explaining the loss of timezone info and the recommended alternatives.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org