You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2026/07/31 23:06:28 UTC
[PR] [SPARK-58489] Support datetime functions in `DateTimeFunctions` [spark-connect-swift]
dongjoon-hyun opened a new pull request, #462:
URL: https://github.com/apache/spark-connect-swift/pull/462
### What changes were proposed in this pull request?
This PR aims to add datetime functions in a new file, `DateTimeFunctions.swift`. The function names and signatures follow Scala's `org.apache.spark.sql.functions` (`datetime_funcs` group).
| Category | Functions |
| --- | --- |
| Current date/time | `curdate`, `current_date`, `current_time` (with a `precision` overload), `current_timestamp`, `current_timezone`, `localtimestamp`, `now`, `unix_timestamp()` |
| Field extraction | `year`, `quarter`, `month`, `day`, `dayofmonth`, `dayofweek`, `dayofyear`, `hour`, `minute`, `second`, `weekday`, `weekofyear`, `dayname`, `monthname`, `extract`, `date_part`, `datepart` |
| Date arithmetic | `add_months`, `date_add`/`dateadd`, `date_sub`, `datediff`/`date_diff`, `months_between` (with a `roundOff` overload), `next_day`, `last_day`, `trunc`, `date_trunc`, `timestamp_add`, `timestamp_diff` |
| Construction | `make_date`, `make_time` |
| Conversion and parsing | `to_date`/`try_to_date`, `to_timestamp`/`try_to_timestamp`, `to_timestamp_ltz`, `to_timestamp_ntz`, `to_time`/`try_to_time`, `date_format`, `from_unixtime`, `unix_timestamp(column)`, `to_unix_timestamp` |
| Epoch conversion | `date_from_unix_date`, `unix_date`, `unix_seconds`, `unix_millis`, `unix_micros`, `unix_nanos`, `timestamp_seconds`, `timestamp_millis`, `timestamp_micros`, `timestamp_nanos` |
| Time zone | `from_utc_timestamp`, `to_utc_timestamp`, `convert_timezone` (2-arg and 3-arg) |
| TIME type | `time_bucket`, `time_diff`, `time_trunc`, `time_from_seconds`, `time_from_millis`, `time_from_micros`, `time_to_seconds`, `time_to_millis`, `time_to_micros` |
| Windowing | `window`, `window_time`, `session_window` |
In total, 101 public functions (76 distinct names) are added. Following Scala, some Swift APIs map to different SQL function names.
| Swift API | SQL function |
| --- | --- |
| `timestamp_add` | `timestampadd` |
| `timestamp_diff` | `timestampdiff` |
| `unix_timestamp()` | `unix_timestamp(current_timestamp())` |
The following `datetime_funcs` families are excluded intentionally and can be added in a follow-up.
| Excluded | Reason |
| --- | --- |
| `make_dt_interval` (5 overloads), `make_interval` (7), `try_make_interval` (7), `make_ym_interval` (3) | Interval construction family with a large overload surface |
| `make_timestamp` (4), `try_make_timestamp` (4), `make_timestamp_ltz` (2), `try_make_timestamp_ltz` (2), `make_timestamp_ntz` (2), `try_make_timestamp_ntz` (2) | Timestamp construction family with many 6~7-parameter overloads |
There are no deprecated functions and no `(columnName: String)`-only convenience overloads in this group.
### Why are the changes needed?
To improve the usability of the column-based `DataFrame` API by providing datetime functions in the same shape as Scala/PySpark, so users can write queries without falling back to `selectExpr` or raw SQL strings.
### Does this PR introduce _any_ user-facing change?
Yes, this adds new public functions. There is no behavior change to existing APIs.
### How was this patch tested?
Pass the CIs with a newly added test suite.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58489] Support datetime functions in `DateTimeFunctions` [spark-connect-swift]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #462:
URL: https://github.com/apache/spark-connect-swift/pull/462#issuecomment-5148843676
Thank you, @viirya !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58489] Support datetime functions in `DateTimeFunctions` [spark-connect-swift]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #462: [SPARK-58489] Support datetime functions in `DateTimeFunctions`
URL: https://github.com/apache/spark-connect-swift/pull/462
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58489] Support datetime functions in `DateTimeFunctions` [spark-connect-swift]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #462:
URL: https://github.com/apache/spark-connect-swift/pull/462#issuecomment-5148845311
Merged to main
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org