You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2026/03/30 18:01:02 UTC
[PR] Add missing datetime functions [datafusion-python]
timsaucer opened a new pull request, #1467:
URL: https://github.com/apache/datafusion-python/pull/1467
# Which issue does this PR close?
Closes #1451
# Rationale for this change
Adds missing functions
# What changes are included in this PR?
Expose rust code.
Add python wrapper.
Add unit tests.
# Are there any user-facing changes?
New addition only
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing datetime functions [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer merged PR #1467:
URL: https://github.com/apache/datafusion-python/pull/1467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing datetime functions [datafusion-python]
Posted by "nuno-faria (via GitHub)" <gi...@apache.org>.
nuno-faria commented on code in PR #1467:
URL: https://github.com/apache/datafusion-python/pull/1467#discussion_r3028862789
##########
python/datafusion/functions.py:
##########
@@ -1948,6 +1951,15 @@ def now() -> Expr:
return Expr(f.now())
+def current_timestamp() -> Expr:
+ """Returns the current timestamp in nanoseconds.
+
+ See Also:
+ This is an alias for :py:func:`now`.
+ """
+ return now()
+
+
Review Comment:
Don't know if it's critical but the resulting column name in this case will be `now`, while in DataFusion it's `current_timestamp`.
```sql
> select now();
+-------------------------------+
| now() |
+-------------------------------+
| 2026-04-02T15:46:14.652469200 |
+-------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
> select current_timestamp();
+----------------------------+
| current_timestamp() |
+----------------------------+
| 2026-04-02T15:46:17.026273 |
+----------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
```
Should this function call the Rust equivalent directly?
##########
python/datafusion/functions.py:
##########
@@ -1970,6 +1982,15 @@ def to_char(arg: Expr, formatter: Expr) -> Expr:
return Expr(f.to_char(arg.expr, formatter.expr))
+def date_format(arg: Expr, formatter: Expr) -> Expr:
+ """Returns a string representation of a date, time, timestamp or duration.
+
+ See Also:
+ This is an alias for :py:func:`to_char`.
+ """
+ return to_char(arg, formatter)
+
+
Review Comment:
Similar issue here:
```sql
> select to_char('2026-12-13'::timestamp, '%Y/%m/%d');
+----------------------------------------------+
| to_char(Utf8("2026-12-13"),Utf8("%Y/%m/%d")) |
+----------------------------------------------+
| 2026/12/13 |
+----------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
> select date_format('2026-12-13'::timestamp, '%Y/%m/%d');
+-------------------------------------------------------------------------+
| date_format(CAST(Utf8("2026-12-13") AS Timestamp(ns)),Utf8("%Y/%m/%d")) |
+-------------------------------------------------------------------------+
| 2026/12/13 |
+-------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing datetime functions [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on code in PR #1467:
URL: https://github.com/apache/datafusion-python/pull/1467#discussion_r3029905116
##########
python/datafusion/functions.py:
##########
@@ -1970,6 +1982,15 @@ def to_char(arg: Expr, formatter: Expr) -> Expr:
return Expr(f.to_char(arg.expr, formatter.expr))
+def date_format(arg: Expr, formatter: Expr) -> Expr:
+ """Returns a string representation of a date, time, timestamp or duration.
+
+ See Also:
+ This is an alias for :py:func:`to_char`.
+ """
+ return to_char(arg, formatter)
+
+
Review Comment:
I think this is okay for aliases. If you think it's a blocker we can open an issue to try to overcome this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing datetime functions [datafusion-python]
Posted by "nuno-faria (via GitHub)" <gi...@apache.org>.
nuno-faria commented on code in PR #1467:
URL: https://github.com/apache/datafusion-python/pull/1467#discussion_r3030398924
##########
python/datafusion/functions.py:
##########
@@ -1970,6 +1982,15 @@ def to_char(arg: Expr, formatter: Expr) -> Expr:
return Expr(f.to_char(arg.expr, formatter.expr))
+def date_format(arg: Expr, formatter: Expr) -> Expr:
+ """Returns a string representation of a date, time, timestamp or duration.
+
+ See Also:
+ This is an alias for :py:func:`to_char`.
+ """
+ return to_char(arg, formatter)
+
+
Review Comment:
Not a problem for me as well. Just pointing it out just in case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add missing datetime functions [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on code in PR #1467:
URL: https://github.com/apache/datafusion-python/pull/1467#discussion_r3029902701
##########
python/datafusion/functions.py:
##########
@@ -1970,6 +1982,15 @@ def to_char(arg: Expr, formatter: Expr) -> Expr:
return Expr(f.to_char(arg.expr, formatter.expr))
+def date_format(arg: Expr, formatter: Expr) -> Expr:
+ """Returns a string representation of a date, time, timestamp or duration.
+
+ See Also:
+ This is an alias for :py:func:`to_char`.
+ """
+ return to_char(arg, formatter)
+
+
Review Comment:
I was looking at this. It's true that we do get the alias via SQL, but we actually will get this same behavior for all aliases. This impacts our existing things like:
```python
from datafusion import SessionContext, functions as F, col
import pyarrow as pa
import numpy as np
data = [[1.0, 2.0, 3.0, 3.0], [4.0, 5.0, 3.0], [6.0]]
ctx = SessionContext()
batch = pa.RecordBatch.from_arrays([np.array(data, dtype=object)], names=["arr"])
df = ctx.create_dataframe([[batch]])
df.select(
F.list_sort(col("arr")),
).show()
```
Results in:
```
DataFrame()
+----------------------------------------------------------------------------------+
| array_sort(c3b76956cf32e4475aab41190fde60912.arr,Utf8("ASC"),Utf8("NULLS LAST")) |
+----------------------------------------------------------------------------------+
| [1.0, 2.0, 3.0, 3.0] |
| [3.0, 4.0, 5.0] |
| [6.0] |
+----------------------------------------------------------------------------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org