You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/06/29 16:56:38 UTC
[I] Evaluate moving from chrono to Jiff [datafusion-comet]
andygrove opened a new issue, #4754:
URL: https://github.com/apache/datafusion-comet/issues/4754
### What is the problem the feature request solves?
Chrono is being deprecated - https://github.com/chronotope/chrono-tz/issues/231
Recommendation is to use [Jiff](https://crates.io/crates/jiff) instead.
Let's explore this.
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Evaluate moving from chrono to Jiff [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #4754:
URL: https://github.com/apache/datafusion-comet/issues/4754#issuecomment-6471504763
I am no longer working on this issue, so I have unassigned myself. It is open for another contributor to pick up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Evaluate moving from chrono to Jiff [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #4754:
URL: https://github.com/apache/datafusion-comet/issues/4754#issuecomment-4835284249
I ran a small spike to scope this concretely, converting one timezone-free expression (`next_day`) from chrono to jiff end to end: #4755. Sharing what we learned and a suggested approach for the broader work.
### Key learnings
1. **We do not need to wait on arrow-rs.** The Arrow boundary for date/time columns is raw integers (`Date32Array::value` returns `i32` epoch days; timestamps are `i64`). Our expression code can build jiff types directly from those integers and skip Arrow's chrono-returning helpers (`Date32Type::to_naive_date_opt`, `date32_to_datetime`). So Comet's own code can move to jiff today.
2. **chrono will remain in the dependency tree until arrow-rs migrates**, because `arrow` depends on chrono and pulls `chrono-tz` via its feature. Moving Comet to jiff therefore buys API ergonomics and a single date/time library in our code, but does not shrink the dependency tree on its own. Full chrono removal is a separate downstream goal gated on upstream arrow-rs.
3. **jiff keeps epoch-day conversion private on purpose** (`from_unix_epoch_day` / `to_unix_epoch_day` are crate-private). The public idiom is an anchor date plus a day `Span`:
- epoch day to date: `Date::constant(1970, 1, 1).checked_add(days.days())`
- date to epoch day: `epoch.until(date).get_days()`
This is slightly more verbose than chrono, so a tiny shared helper module is worth adding up front.
4. **jiff civil dates span years -9999..=9999**, narrower than chrono and Date32. Out-of-range values should map to NULL (matching today's nullable behavior). Not a concern for realistic Spark data, but worth a conscious decision per expression.
5. **The real cost is the timezone path, not the calendar math.** Date-only expressions are a near-mechanical translation (chrono `Weekday::days_since` maps 1:1 to jiff `Weekday::since`, etc.). Timestamp and timezone expressions are where the work is: they need jiff's tzdb features and a rework of `timezone.rs` (currently chrono-tz based), plus careful comparison of jiff's tz behavior against Spark and chrono-tz.
### Suggested approach
- **Phase 1 (low risk):** migrate timezone-free date expressions (`next_day`, `dayname`/`monthname`, `make_date`, ...) behind a shared `epoch_day <-> jiff::civil::Date` helper. Use `jiff = { default-features = false, features = ["std"] }` for these since the tzdb bundles are not needed.
- **Phase 2:** migrate timestamp/timezone expressions, enabling jiff's tzdb features and reworking `timezone.rs`. Validate tz behavior against Spark.
- **Phase 3:** track arrow-rs for chrono removal; only then can chrono leave `Cargo.lock`.
Each phase is independently shippable, and Phase 1 can be split per expression. The spike in #4755 is a working example of the Phase 1 pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Evaluate moving from chrono to Jiff [datafusion-comet]
Posted by "BurntSushi (via GitHub)" <gi...@apache.org>.
BurntSushi commented on issue #4754:
URL: https://github.com/apache/datafusion-comet/issues/4754#issuecomment-4844059941
I'm the author of Jiff and I'd be happy to answer any questions if you have them! Especially around time zones, I should be able to clarify any points of confusion around the differences between Jiff and `chrono-tz`. They are quite different at the API level (with Jiff not having a `TimeZone` trait), but Jiff's core behavior should be a strict superset of `chrono`. (The most notable thing that Jiff handles in a strictly more flexible way than `chrono` are gaps in civil datetimes when a time zone transition moves from a smaller UTC offset to a bigger UTC offset. i.e., Typically when a region enters daylight saving time.)
> jiff keeps epoch-day conversion private on purpose (from_unix_epoch_day / to_unix_epoch_day are crate-private). The public idiom is an anchor date plus a day Span:
It's likely that this will change with a [new `jiff::civil::UnixEpochDay` type](https://github.com/BurntSushi/jiff/issues/595). I imagine this is quite useful if you want to represent dates as 32-bit integers. Without it, I think you have to roll your own math to convert a Gregorian date to a single day number. Jiff really should own that math because it's quite performance critical and easy to get wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Evaluate moving from chrono to Jiff [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #4754:
URL: https://github.com/apache/datafusion-comet/issues/4754#issuecomment-4861125004
> I'm the author of Jiff and I'd be happy to answer any questions if you have them! Especially around time zones, I should be able to clarify any points of confusion around the differences between Jiff and `chrono-tz`. They are quite different at the API level (with Jiff not having a `TimeZone` trait), but Jiff's core behavior should be a strict superset of `chrono`. (The most notable thing that Jiff handles in a strictly more flexible way than `chrono` are gaps in civil datetimes when a time zone transition moves from a smaller UTC offset to a bigger UTC offset. i.e., Typically when a region enters daylight saving time.)
>
> > jiff keeps epoch-day conversion private on purpose (from_unix_epoch_day / to_unix_epoch_day are crate-private). The public idiom is an anchor date plus a day Span:
>
> It's likely that this will change with a [new `jiff::civil::UnixEpochDay` type](https://github.com/BurntSushi/jiff/issues/595). I imagine this is quite useful if you want to represent dates as 32-bit integers. Without it, I think you have to roll your own math to convert a Gregorian date to a single day number. Jiff really should own that math because it's quite performance critical and easy to get wrong.
Thanks @BurntSushi! I appreciate the offer. We'll try and figure things out ourselves, but will definitely ask for help if we hit anything the LLMs can't manage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org