You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "koopatroopa787 (via GitHub)" <gi...@apache.org> on 2026/06/03 15:45:33 UTC
[PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
koopatroopa787 opened a new pull request, #22734:
URL: https://github.com/apache/datafusion/pull/22734
## Which issue does this PR close?
Closes #22717
## Rationale for this change
Spark's `next_day` function performs exact (case-insensitive) matching on the `dayOfWeek` argument. Whitespace-padded values like `' MO '` are not valid day names in Spark and should return `NULL`. DataFusion was calling `.trim()` before matching, so `' MO '` was silently accepted and returned a date — diverging from Spark's behavior.
## What changes are included in this PR?
- `datafusion/spark/src/function/datetime/next_day.rs`: removed `.trim()` from the `spark_next_day` helper so whitespace-padded day names fall through to the `None` branch.
- `datafusion/sqllogictest/test_files/spark/datetime/next_day.slt`: added three regression tests (`' MO '`, `' Monday '`, `' sun '`) that all now correctly return `NULL`.
## Are there any user-facing changes?
Yes — `next_day(date, ' MO ')` now returns `NULL` instead of a date. This is a bug fix that aligns with Spark's actual behavior.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal commented on PR #22734:
URL: https://github.com/apache/datafusion/pull/22734#issuecomment-4614708603
@koopatroopa787 another PR has already been approved for this issue #22720
I would appreciate if you first check if there are open PRs for a particular issue before moving forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
Posted by "koopatroopa787 (via GitHub)" <gi...@apache.org>.
koopatroopa787 closed pull request #22734: fix(spark): next_day should not trim whitespace from dayOfWeek
URL: https://github.com/apache/datafusion/pull/22734
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
Posted by "koopatroopa787 (via GitHub)" <gi...@apache.org>.
koopatroopa787 commented on PR #22734:
URL: https://github.com/apache/datafusion/pull/22734#issuecomment-4614742006
Hey @kumarUjjawal, really sorry about this — I missed #22720 when I checked the issue. Closing this one out. Thanks for the heads up, I'll make sure to check for existing PRs before opening one next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal commented on PR #22734:
URL: https://github.com/apache/datafusion/pull/22734#issuecomment-4615137204
> Hey @kumarUjjawal, really sorry about this — I missed #22720 when I checked the issue. Closing this one out. Thanks for the heads up, I'll make sure to check for existing PRs before opening one next time.
Thank you! Appreciate it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix(spark): next_day should not trim whitespace from dayOfWeek [datafusion]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #22734:
URL: https://github.com/apache/datafusion/pull/22734#discussion_r3349925711
##########
datafusion/spark/src/function/datetime/next_day.rs:
##########
@@ -210,7 +210,7 @@ where
fn spark_next_day(days: i32, day_of_week: &str) -> Option<i32> {
let date = Date32Type::to_naive_date_opt(days)?;
- let day_of_week = day_of_week.trim().to_uppercase();
+ let day_of_week = day_of_week.to_uppercase();
Review Comment:
`to_uppercase()` performs full Unicode uppercasing and allocates a new `String`. Since Spark day-name tokens are ASCII-only, consider using an ASCII-only approach (e.g., `to_ascii_uppercase()` or matching via `eq_ignore_ascii_case` without allocation) to reduce overhead and make the behavior more strictly aligned to Spark’s expected token set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org