You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2026/09/29 02:36:02 UTC
[I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
viirya opened a new issue, #25855:
URL: https://github.com/apache/datafusion/issues/25855
### Describe the bug
With a month stride, `date_bin` bins through chrono's `DateTime<Utc>`, whose range (about ±262,000 years) is much narrower than `Timestamp(Second)`, `Timestamp(Millisecond)` or `Timestamp(Microsecond)`. Values outside it return `NULL`, although their bin can be represented in the output type.
This is the same kind of limit as the i64 nanosecond overflow fixed by #25815: a limit of the intermediate representation rather than of the output type, only much narrower. Fixed-duration strides are not affected.
### To Reproduce
On #25815 (`10000000000000` seconds is about the year 318,857):
```sql
-- Month stride: NULL
select arrow_cast(date_bin(interval '1 month', arrow_cast(10000000000000, 'Timestamp(Second)')), 'Int64');
-- NULL
-- Day stride on the same value works
select arrow_cast(date_bin(interval '1 day', arrow_cast(10000000000000, 'Timestamp(Second)')), 'Int64');
-- 9999999936000
```
(The `Int64` cast is only needed because values outside chrono's range cannot be displayed as dates.)
### Expected behavior
The month-stride query returns the start of the month containing the value, like the day-stride query does.
### Additional context
The month arithmetic in `bin_months` / `date_bin_months_interval_wide` (`datafusion/functions/src/datetime/date_bin.rs`) could use i64 day counts instead of chrono, for example with the `civil_from_days` / `days_from_civil` helpers in `date_trunc.rs`. Then every remaining `NULL` would be a bin that is outside the output type.
Follow-up from #25815.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-5882566952
We'll work on this after #25815 merges.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6075191122
No problem. Thanks for being interested in contributing to DataFusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "ThilakShekharShriyan (via GitHub)" <gi...@apache.org>.
ThilakShekharShriyan commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6075024997
Sorry about that, @viirya. I did see your comment that you were planning to work on this after #25815, and I should have asked before commenting take. I thought the issue was still open to pick up. I didn't mean to overlook your note or step on your work.
I'm happy to leave this with you if you still want it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "ThilakShekharShriyan (via GitHub)" <gi...@apache.org>.
ThilakShekharShriyan commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6075088385
Thanks I'll Take it , apologies again :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "ThilakShekharShriyan (via GitHub)" <gi...@apache.org>.
ThilakShekharShriyan commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6074898315
take
Month strides will be binned with the integer civil-date helpers already used by `date_trunc`, instead of `DateTime<Utc>`. A second, millisecond, or microsecond timestamp outside chrono's range then returns the month bin when that bin fits in the output type, and NULL only when it does not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6075036180
Thanks @ThilakShekharShriyan . Feel free to take this if you want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] date_bin: month strides on s/ms/us timestamps return NULL outside the chrono DateTime range [datafusion]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #25855:
URL: https://github.com/apache/datafusion/issues/25855#issuecomment-6074971753
I am okay if other is interested in working on this. But as I have already left a comment saying we will work on this, maybe you should ask if you can take this instead of pretending that you don't see it. @ThilakShekharShriyan Do you agree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org