You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "icexelloss (via GitHub)" <gi...@apache.org> on 2025/08/21 20:59:33 UTC
[I] Support Day-light-saving aware shifting [arrow]
icexelloss opened a new issue, #47400:
URL: https://github.com/apache/arrow/issues/47400
### Describe the enhancement requested
In Pandas, there is a way to do "day offset shift" that is day light saving aware, e.g.
```
import pyarrow as pa
import pandas as pd
import pyarrow.compute as pc
begin = pd.Timestamp('20250308 9:30', tz='America/New_York')
print(begin)
print(begin + pd.DateOffset(days=1))
print(begin + pd.DateOffset(days=2))
```
Outputs:
```
2025-03-08 09:30:00-05:00
2025-03-09 09:30:00-04:00
2025-03-10 09:30:00-04:00
```
However, there is no way to do such operations as far as I am aware with arrow compute, the closest I found is this
```
begin = pa.scalar(pd.Timestamp('20250309', tz='America/New_York'))
pc.add(begin, pa.scalar((0, 1, 0), type=pa.month_day_nano_interval()))
```
However, it seems not supported, I also didn't find any specific arrow compute function to do this.
IMO this would be really useful to deal with time in non-UTC timezone.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "jasonbates692-spec (via GitHub)" <gi...@apache.org>.
jasonbates692-spec commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3266444389
I'm Jason Bates trying to get my money from this account for my phone number so I'm not sure what up where you at I got to come to my bank account for my money from this company to come over and take care of my money from this company to come to the first one in the morning but I was able to find a few things for the office to see if I could get my phone number for the first time in a few days or so but it will probably take me to the first time in my life to get a refund on my way to get a chance to come to the office if you want me to come over and take you to get a chance to see if you can get a refund from this person to come to your place and get it will probably be there in about 10 minutes or so but I will probably go back to the house so I'll see you then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3233897304
@pitrou I don't think we currently support this, but we definitely should. Currently one could perhaps use temporal rounding as a workaround, so something like `pc.round(pc.add(zoned_timestamps + some_duration), unit='day')` (doing this from memory, it's likely wrong API-wise). But a proper "`add_temporal_unit`" kernel would be nicer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-5601008942
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "icexelloss (via GitHub)" <gi...@apache.org>.
icexelloss commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-5794202441
Hi Rok!
Yes I definitely think this would still be very useful. It is currently an
annoying issue that we cannot use Arrow kernel to perform some timestamp
logic so we had to round trip to pandas and do operation such as “add one
day”.
On Wed, Sep 23, 2026 at 7:26 AM Rok Mihevc ***@***.***> wrote:
> *rok* left a comment (apache/arrow#47400)
> <https://github.com/apache/arrow/issues/47400#issuecomment-5793973262>
>
> @icexelloss <https://github.com/icexelloss> is this still something you
> need and should be reopened?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/arrow/issues/47400?email_source=notifications&email_token=AAGBXLB5S5LG4JJBLRJLZ235QOXO5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNZZGM4TOMZSGYZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-5793973262>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAGBXLEDRNGWSQGDP4P2B4D5QOXO5AVCNFSNUABEKJSXA33TNF2G64TZHM2TCOJQGUZTKMZ3JFZXG5LFHMZTGNBTGE3TEMRUGGQXMAQ>
> .
> Triage notifications, keep track of coding agent tasks and review pull
> requests on the go with GitHub Mobile for iOS
> <https://github.com/notifications/mobile/ios/AAGBXLEFWJ6JPGU5EDMCO735QOXO5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNZZGM4TOMZSGYZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y>
> and Android
> <https://github.com/notifications/mobile/android/AAGBXLBYZ6KDNGAL26IB3VD5QOXO5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNZZGM4TOMZSGYZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>.
> Download it today!
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3233661564
I think we can follow Pandas' lead where days and larger units are considered logical while hours and smaller units are considered physical:
```python
>>> begin
Timestamp('2025-03-08 09:30:00-0500', tz='America/New_York')
# Logical calendar addition
>>> begin + pd.DateOffset(days=2)
Timestamp('2025-03-10 09:30:00-0400', tz='America/New_York')
>>> begin + pd.DateOffset(months=1)
Timestamp('2025-04-08 09:30:00-0400', tz='America/New_York')
# Physical duration addition
>>> begin + pd.DateOffset(hours=48)
Timestamp('2025-03-10 10:30:00-0400', tz='America/New_York')
>>> begin + pd.DateOffset(seconds=48*3600)
Timestamp('2025-03-10 10:30:00-0400', tz='America/New_York')
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3266407375
Also, we should be careful to follow Pandas' logic when adding/subtracting a mixed interval:
```python
>>> end = begin + pd.DateOffset(days=1, hours=24)
>>> end - pd.DateOffset(days=1, hours=24) == begin
True
>>> end == begin + pd.DateOffset(days=1) + pd.DateOffset(hours=24)
True
>>> end == begin + pd.DateOffset(hours=24) + pd.DateOffset(days=1)
False
>>> end - pd.DateOffset(days=1) - pd.DateOffset(hours=24) == begin
False
>>> end - pd.DateOffset(hours=24) - pd.DateOffset(days=1) == begin
True
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3266707540
I just recalled we do have several [temporal subtraction kernels](https://arrow.apache.org/docs/python/api/compute.html#temporal-difference) that (I believe) look calendar units for units greater than hour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #47400: [C++] Support DST-aware timestamp + interval addition
URL: https://github.com/apache/arrow/issues/47400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-3233629833
@rok Did we have any plans to add this? Are there any difficulties to know of?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-5793973262
@icexelloss is this still something you need and should be reopened?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support DST-aware timestamp + interval addition [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #47400:
URL: https://github.com/apache/arrow/issues/47400#issuecomment-5794414108
@icexelloss great to know, let's reopen this and see if someone contributes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org