You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "buraksenn (via GitHub)" <gi...@apache.org> on 2026/08/01 22:01:15 UTC

[PR] minor(fix): correct to_date results for formatted pre-epoch datetimes [datafusion]

buraksenn opened a new pull request, #24049:
URL: https://github.com/apache/datafusion/pull/24049

   ## Which issue does this PR close?
   
   - Closes N\A but I can open an issue
   
    ## Rationale for this change
   
    Formatted `to_date` inputs are parsed as milliseconds since the Unix epoch and then converted to `Date32` days. Integer division truncates toward zero, so negative sub-day timestamps are incorrectly mapped to day `0` instead of day `-1`.
   
    ```sql
    -- DataFusion before this change
    SELECT to_date('1969-12-31 12:00:00', '%Y-%m-%d %H:%M:%S');
    -- 1970-01-01
   
    -- PostgreSQL
    SELECT to_date('1969-12-31 12:00:00', 'YYYY-MM-DD HH24:MI:SS');
    -- 1969-12-31
   
    -- DuckDB
    SELECT CAST(strptime('1969-12-31 12:00:00', '%Y-%m-%d %H:%M:%S') AS DATE);
    -- 1969-12-31
   ```
   
   ## What changes are included in this PR?
   
   Use Euclidean division when converting formatted timestamp milliseconds to days, preserving the correct date for timestamps before the Unix epoch.
   
   ## Are these changes tested?
   Yes new slt test for this case.
   
   ## Are there any user-facing changes?
   
   No API changes. Formatted pre-epoch datetimes now return the correct date.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] minor(fix): correct to_date results for formatted pre-epoch datetimes [datafusion]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24049:
URL: https://github.com/apache/datafusion/pull/24049#issuecomment-5153748103

   ## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/24049?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 80.86%. Comparing base ([`455a3ad`](https://app.codecov.io/gh/apache/datafusion/commit/455a3add52d051a20df9960a726ee9acb98528a3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d54bdf`](https://app.codecov.io/gh/apache/datafusion/commit/6d54bdfbbbcf71251ff2ea83136bab126550b9c6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 26 commits behind head on main.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff             @@
   ##             main   #24049      +/-   ##
   ==========================================
   + Coverage   80.84%   80.86%   +0.01%     
   ==========================================
     Files        1096     1101       +5     
     Lines      373832   375446    +1614     
     Branches   373832   375446    +1614     
   ==========================================
   + Hits       302240   303587    +1347     
   - Misses      53563    53767     +204     
   - Partials    18029    18092      +63     
   ```
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/24049?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] minor(fix): correct to_date results for formatted pre-epoch datetimes [datafusion]

Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal commented on PR #24049:
URL: https://github.com/apache/datafusion/pull/24049#issuecomment-5157073395

   Thank you @buraksenn 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] minor(fix): correct to_date results for formatted pre-epoch datetimes [datafusion]

Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal merged PR #24049:
URL: https://github.com/apache/datafusion/pull/24049


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org