You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/08/02 10:09:51 UTC
(datafusion) branch gh-readonly-queue/main/pr-24049-60d9b67ec9428cac3e5b7ec7f25774f797063b65 created (now eb8e38e1de)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-24049-60d9b67ec9428cac3e5b7ec7f25774f797063b65
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at eb8e38e1de minor(fix): correct to_date results for formatted pre-epoch datetimes (#24049)
This branch includes the following new commits:
new eb8e38e1de minor(fix): correct to_date results for formatted pre-epoch datetimes (#24049)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: minor(fix): correct to_date results for formatted pre-epoch datetimes (#24049)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-24049-60d9b67ec9428cac3e5b7ec7f25774f797063b65
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit eb8e38e1de7322fe52501323648d78c8d3bc3e43
Author: Burak Şen <bu...@gmail.com>
AuthorDate: Sun Aug 2 13:09:29 2026 +0300
minor(fix): correct to_date results for formatted pre-epoch datetimes (#24049)
## Which issue does this PR close?
- Closes N\A but I can open an issue
## Rationale for this change
Formatted `to_date` inputs are parsed as milliseconds since the Unix
epoch and then converted to `Date32` days. Integer division truncates
toward zero, so negative sub-day timestamps are incorrectly mapped to
day `0` instead of day `-1`.
```sql
-- DataFusion before this change
SELECT to_date('1969-12-31 12:00:00', '%Y-%m-%d %H:%M:%S');
-- 1970-01-01
-- PostgreSQL
SELECT to_date('1969-12-31 12:00:00', 'YYYY-MM-DD HH24:MI:SS');
-- 1969-12-31
-- DuckDB
SELECT CAST(strptime('1969-12-31 12:00:00', '%Y-%m-%d %H:%M:%S') AS
DATE);
-- 1969-12-31
```
## What changes are included in this PR?
Use Euclidean division when converting formatted timestamp milliseconds to days, preserving the correct date for timestamps before the Unix epoch.
## Are these changes tested?
Yes new slt test for this case.
## Are there any user-facing changes?
No API changes. Formatted pre-epoch datetimes now return the correct date.
---
datafusion/functions/src/datetime/to_date.rs | 2 +-
datafusion/sqllogictest/test_files/datetime/dates.slt | 6 ++++++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/datafusion/functions/src/datetime/to_date.rs b/datafusion/functions/src/datetime/to_date.rs
index ed5b8b1632..e0a14e056a 100644
--- a/datafusion/functions/src/datetime/to_date.rs
+++ b/datafusion/functions/src/datetime/to_date.rs
@@ -100,7 +100,7 @@ impl ToDateFunc {
args,
|s, format| {
string_to_timestamp_millis_formatted(s, format)
- .map(|n| n / (24 * 60 * 60 * 1_000))
+ .map(|n| n.div_euclid(24 * 60 * 60 * 1_000))
.and_then(|v| {
v.try_into().map_err(|_| {
internal_datafusion_err!("Unable to cast to Date32 for converting from i64 to i32 failed")
diff --git a/datafusion/sqllogictest/test_files/datetime/dates.slt b/datafusion/sqllogictest/test_files/datetime/dates.slt
index abf92e1565..a6a5f480f7 100644
--- a/datafusion/sqllogictest/test_files/datetime/dates.slt
+++ b/datafusion/sqllogictest/test_files/datetime/dates.slt
@@ -139,6 +139,12 @@ SELECT to_date('01-14-2023 01:01:30+05:30', '%q', '%d-%m-%Y %H/%M/%S', '%+', '%m
----
2023-01-13
+# Formatted pre-epoch datetimes retain their calendar date
+query D
+SELECT to_date('1969-12-31 12:00:00', '%Y-%m-%d %H:%M:%S');
+----
+1969-12-31
+
statement error DataFusion error: Execution error: to_date function unsupported data type at index 1: List
SELECT to_date('2022-08-03T14:38:50+05:30', make_array('%s', '%q', '%d-%m-%Y %H:%M:%S%#z', '%+'));
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org